-- Logs begin at Thu 2022-07-28 00:20:18 UTC, end at Thu 2022-07-28 01:32:40 UTC. -- Jul 28 00:58:31.504945 dv-cinder-815582-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 00:58:31.516333 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 00:58:31 2022] *** Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: nodename: dv-cinder-815582-11-iscsi Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: machine: x86_64 Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: clock source: unix Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: pcre jit disabled Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: detected number of CPU cores: 4 Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: current working directory: / Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: detected binary path: /usr/bin/uwsgi-core Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: your processes number limit is 47680 Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: your memory page size is 4096 bytes Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: detected max file descriptor number: 2048 Jul 28 00:58:31.520282 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: lock engine: pthread robust mutexes Jul 28 00:58:31.521885 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: thunder lock: enabled Jul 28 00:58:31.521885 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 00:58:31.521885 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 00:58:31.561297 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: Python main interpreter initialized at 0x562fe70fc9c0 Jul 28 00:58:31.561297 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: python threads support enabled Jul 28 00:58:31.561297 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: your server socket listen backlog is limited to 100 connections Jul 28 00:58:31.561297 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: your mercy for graceful operations on workers is 90 seconds Jul 28 00:58:31.561834 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 00:58:31.561834 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: *** Operational MODE: preforking *** Jul 28 00:58:31.562054 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: *** uWSGI is running in multiple interpreter mode *** Jul 28 00:58:31.562054 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: spawned uWSGI master process (pid: 80406) Jul 28 00:58:31.562343 dv-cinder-815582-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 00:58:31.566870 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: spawned uWSGI worker 1 (pid: 80415, cores: 1) Jul 28 00:58:31.566870 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: spawned uWSGI worker 2 (pid: 80416, cores: 1) Jul 28 00:58:31.566870 dv-cinder-815582-11-iscsi devstack@keystone.service[80406]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 00:58:37.135792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 00:58:37.135792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: from cryptography.utils import int_from_bytes Jul 28 00:58:37.136708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 00:58:37.136708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: from cryptography.utils import int_from_bytes Jul 28 00:58:37.140320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 00:58:37.140320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: from cryptography.utils import int_from_bytes Jul 28 00:58:37.140852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 00:58:37.140852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: from cryptography.utils import int_from_bytes Jul 28 00:58:37.186726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 00:58:37.186991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 00:58:37.187141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] command line args: [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 00:58:37.187556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 00:58:37.187821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ================================================================================ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 00:58:37.188238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] admin_token = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.188668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_dir = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.188949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.189099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 00:58:37.189291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_source = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.189448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 00:58:37.189579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] command line args: [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 00:58:37.189703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] debug = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.189830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 00:58:37.189949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ================================================================================ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 00:58:37.190270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] admin_token = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_dir = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_source = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.190998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] debug = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.191378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.191536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.191662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.191662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.191869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.191977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_config_append = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_config_append = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.192887 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_dir = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_dir = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.193942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.194221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.194350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.194485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.194485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.194730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.194912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.195963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.196084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.196222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.196399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.196530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.196815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.196952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.197110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] publish_errors = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.197233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.197367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.197534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.197669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.197872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] standard_threads = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.198906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_journal = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_json = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_stderr = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.199889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_syslog = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.200238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.200238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.200619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.200759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.200893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] publish_errors = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.201026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.201026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.201264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.201387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.201387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.201613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.201754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.201890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.202024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.202024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.202252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.202375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.202512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.202646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.202770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.202978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] standard_threads = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.203114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.203250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.203390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.203523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.203709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.203840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.203965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.204101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.204242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.204402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_journal = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.204601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.204726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.204848 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_json = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.204971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.205106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_stderr = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.205240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.205365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_syslog = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.205489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.205612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.205729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 00:58:37.205865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.206084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.206214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.206349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.206534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.206671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.206975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.207998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.208181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.208311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.208444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.208656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.208796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.208929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.209964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.210986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.211143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.211290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.211416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.211606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.211735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.211870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.212921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.213046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.213210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.213381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.213517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.213673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.213914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.214203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.214332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.214455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.214581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.214702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.214926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.215913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.216085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.216218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.216384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.216553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.216730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.216901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.217062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.217225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.217386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.217555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.217733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.217916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.218076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.218076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.218423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.218609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.218609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.218940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.219962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.220088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.220213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.220213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.220558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.220699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.220699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.221938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.222906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.223889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.external = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.224891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.password = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.token = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.225947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.226945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.227966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.228120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.228254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.228455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.228580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.228718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.228893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.229034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.229163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.229405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.229549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.229710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.229912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.230205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.230418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.230566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.230708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.230855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.230988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.231213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.231346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.231472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.231679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.231815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.231943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232377 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.232975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.233116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.233331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.233479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.233606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.233734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.233861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.234063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.234194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.234332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.234530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.234712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.234956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.235986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.236195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.236327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.236474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.236656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.236792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.237942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.238071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.238311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.238446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.238573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.238731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.238892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.239978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.240980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.241147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.241277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.241418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.241623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.241757 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.241898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.242088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.242226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.242355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.242513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.242646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.242786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.243973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.244192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.244330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.244457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.244595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.244725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.244852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.245995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.246124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.246251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.246493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.246622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.246746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.246900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.247913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.248908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.249052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.249222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.249357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.249531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.249690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.249822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.external = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.250956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.251951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.252126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.252257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.password = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.252385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.252607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.252741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.token = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.252869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.253879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.254151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.254301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.254443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.254973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.255947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.256170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.256287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.256426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.256685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.256894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.256894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.257268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.257460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.257460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.257832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.258025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.258025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.258401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.258593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.258593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.258989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.259183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.259374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.259565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.259565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.259565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.260122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.260315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.260507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.260712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.260712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.261108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.261314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.261314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.261672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.261865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.261865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.262227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.262428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.262428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.262788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.263049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.263252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.263442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.263442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.263816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.264944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.265303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.265498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.265498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.265884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.266019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.266019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.266434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.266622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.266622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.267904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.268904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.269881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.270887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.271922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.272785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.273900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.274943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.275073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.275073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.275307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.275451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 00:58:37.275581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.275846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.276206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.276611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.276989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.277318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.277646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.277966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.278289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.278622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.278986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.279333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.279657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.279994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.280330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.280656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.280993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.281333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.281652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.281979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.282296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.282626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.282983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.283325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.283649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.283974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.284310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.284645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.284991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.285318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.285656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.285991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.286312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.286635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.287003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.287331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.287658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.288000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.288319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.288651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.288983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.289322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.289642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.289956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.290275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.290594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.290948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.291273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.291605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.291938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.292306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.292786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.293123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.293468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.293805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.294130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.294457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.294780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.295154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.295497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.295833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.296159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.296500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.296815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.297145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.297475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.297798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.298121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.298456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.298781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.299136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.299468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.299794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.300132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.300482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.300805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.301148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.301484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.301805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.302137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.302479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.302819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.303476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.303654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.303955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.304293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.304632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.304969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.305123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.305269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.305397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.305527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.305781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.306119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.306462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.306800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.307171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.307509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.308010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.308347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.308691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.309227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.309553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.309938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.310275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.310616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.310992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.311318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.311661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.311997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.312324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.312652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.312971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.313291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.313626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.313969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.314303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.314531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.314675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.315028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.315348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.315669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.315992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.316657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.317002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.317337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.317657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.317984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.318318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.318654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.319013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.319362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.319686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.320019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.320367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.320684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.321011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.321400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.321746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.322084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.322452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.322824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.323486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.323837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.324209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.324563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.324914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.325405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.325759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.326117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.326485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.327005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.327359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.327754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.328124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.328488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.328836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 00:58:37.329114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 00:58:37.346466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.346934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.354573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.354872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.355098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.363664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.384010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.384255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.400625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.420963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.452204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.452439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.458126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.458372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.519882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.520995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.544988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.545901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.546080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.546205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.546288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.546365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.546438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.557693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.557935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.571011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.571472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.571740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.571873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.572919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.573052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.573167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.573167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 00:58:37.589818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.590025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.590138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 00:58:37.590232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 00:58:37.590378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.590532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.590628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 00:58:37.590730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 00:58:37.590852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.591788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.601231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.601367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.601485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.601592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.601759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.601875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.602037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.602138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.609020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.609152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.613829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.614009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.614090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.614243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.614368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.614466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.614628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.614753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.614925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.615926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.616020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.616162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.616277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.616423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.616536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.639718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.639982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.641221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.641497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.641586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.641791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.646065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.646343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.646509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.646636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.646761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.646919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.653327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.653602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.653802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.653988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.658206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.658421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.658589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.658801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.659897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.660038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.660178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.660178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.660414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.676048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.676285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.676382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.676502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.680321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.680524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.683564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.683741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.683907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.684041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.688316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.688525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.691549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.691741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.694764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.694978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.695900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.696056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.696194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.696323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.712222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.712391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.712516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.712656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.712792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.712929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.713059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.713195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.719910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.720130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.721447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.721669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.721824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.721947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.724499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.724776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.724864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.725057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.725143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.725335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.725423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.725618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.725706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.725912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.737335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.737641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.737780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.737894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.738950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.739091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.739268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.754971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.755471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.757970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.758484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.761628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.762312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.763828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.764381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.765303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.765590 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.765866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.766023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.766207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.766497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.772567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.773049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.774059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.774697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.777094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.777384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.777660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.777757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.777929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.778018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.778285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.778571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.786037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.786332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.786625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.786768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.787038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.787333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.793626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.793889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.794177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.794422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 00:58:37.794619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.794746 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.795030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.795176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.795333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.795422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.795717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.795926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.796117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.796300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.796500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.796767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.796960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.797146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.797329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.797517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.797705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.797945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.798134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.798134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.798507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.798507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.798507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.798507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 00:58:37.831006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562fe70fc9c0 pid: 80416 (default app) Jul 28 00:58:37.831942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562fe70fc9c0 pid: 80415 (default app) Jul 28 00:58:37.834756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d22a4-9122-4b74-8552-5554064da774 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:37.835085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d22a4-9122-4b74-8552-5554064da774 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:37.835365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d22a4-9122-4b74-8552-5554064da774 None None] PATH_INFO: `/v3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:37.839829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jul 28 00:58:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:45.675826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:45.676728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:45.676728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:45.681516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:58:45.687881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:45.688446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:45.688446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:45.758772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: INFO dbcounter [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] Registered counter for database keystone Jul 28 00:58:45.780681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG oslo_db.sqlalchemy.engines [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 00:58:45.787750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writer thread running {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 00:58:45.969773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG passlib.handlers.bcrypt [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80416) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 00:58:45.970201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG passlib.handlers.bcrypt [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80416) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 00:58:46.026273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:46.027926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:46.177399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 00:58:46.312427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:46.384768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: INFO dbcounter [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] Registered counter for database keystone Jul 28 00:58:46.404718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG oslo_db.sqlalchemy.engines [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80415) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 00:58:46.411680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writer thread running {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 00:58:46.535312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3996dbb-4999-4610-9230-363db17fa956 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:46.540731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:46.541415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:46.541705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:46.542152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:46.552061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:46.603481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING keystone.common.rbac_enforcer.enforcer [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 00:58:46.603705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d02d03-aabd-49a8-ad9f-8259c9136259 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:46.614779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 00:58:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 00:58:46.621407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:46.637870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166814d0-6f97-4e18-828d-3682bf2756af None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:46.643004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:46.644171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:46.644462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:46.644754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] PATH_INFO: `/v3/projects/bb3fd2235b214e75887726b6b079e208` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:46.646370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] RBAC: Authorizing `identity:get_project(project_id=bb3fd2235b214e75887726b6b079e208)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:46.696543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING keystone.common.rbac_enforcer.enforcer [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 00:58:46.697249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6aec-70b2-4707-8c65-176113b3f248 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:46.701920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jul 28 00:58:46 2022] GET /identity/v3/projects/bb3fd2235b214e75887726b6b079e208 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:58:48.706027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619c1aa-cbfa-4f0b-9529-240ad192fc54 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:48.706582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619c1aa-cbfa-4f0b-9529-240ad192fc54 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:48.706582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619c1aa-cbfa-4f0b-9529-240ad192fc54 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:48.710779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:48.716896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:48.717408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:48.717408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:48.739303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:48.740848 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:48.749636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 00:58:48.772352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619c1aa-cbfa-4f0b-9529-240ad192fc54 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:48.786613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619c1aa-cbfa-4f0b-9529-240ad192fc54 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:48.791617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6ddca-01d6-41a6-ab21-02c9a155d0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:48.792307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6ddca-01d6-41a6-ab21-02c9a155d0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:48.792590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6ddca-01d6-41a6-ab21-02c9a155d0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:48.792869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6ddca-01d6-41a6-ab21-02c9a155d0c4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:48.801407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6ddca-01d6-41a6-ab21-02c9a155d0c4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:48.803267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6ddca-01d6-41a6-ab21-02c9a155d0c4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:48.813379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jul 28 00:58:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 00:58:48.820450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:48.831745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cd722-4fd4-47c0-9ac8-74cf02bd3885 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:48.837270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516bb80-c47c-45df-9029-9c4e4b2c9f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:48.837945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516bb80-c47c-45df-9029-9c4e4b2c9f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:48.838234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516bb80-c47c-45df-9029-9c4e4b2c9f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:48.838518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516bb80-c47c-45df-9029-9c4e4b2c9f86 admin admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:48.840151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516bb80-c47c-45df-9029-9c4e4b2c9f86 admin admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:48.842024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516bb80-c47c-45df-9029-9c4e4b2c9f86 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:48.843720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jul 28 00:58:48 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:50.925841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aff88-3d88-4038-b0c0-fa743db357ac None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:50.926430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aff88-3d88-4038-b0c0-fa743db357ac None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:50.926430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aff88-3d88-4038-b0c0-fa743db357ac None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:50.928508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:50.934374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:50.934980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:50.934980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:50.960137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:50.961360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:50.963946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94b5ff-b119-4793-80cd-390a8a657a05 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:50.964235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94b5ff-b119-4793-80cd-390a8a657a05 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:50.965432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94b5ff-b119-4793-80cd-390a8a657a05 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:50.965826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:50.971158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6863447-087f-4225-9be4-93f213a146fc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:50.971444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6863447-087f-4225-9be4-93f213a146fc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:50.971726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6863447-087f-4225-9be4-93f213a146fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.018001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG passlib.handlers.bcrypt [None req-d6863447-087f-4225-9be4-93f213a146fc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80415) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 00:58:51.018434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG passlib.handlers.bcrypt [None req-d6863447-087f-4225-9be4-93f213a146fc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80415) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 00:58:51.052160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:58:51.063631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d6863447-087f-4225-9be4-93f213a146fc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:51.065277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d6863447-087f-4225-9be4-93f213a146fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:51.075567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.102869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:51.176814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:58:51.195752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c2ef4-1e23-4838-8cc7-63e81819a2ec None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.198698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6863447-087f-4225-9be4-93f213a146fc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.233797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.234570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.234946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.235308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.247130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:58:51.256714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:58:51.257845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:58:51.259566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:58:51.266665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.320803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6863447-087f-4225-9be4-93f213a146fc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.323565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:51.361529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.362419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.362708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.362986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.380874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:51.401994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0149-963b-4b05-a535-325b056f2f68 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.440916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:51.442383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:51.444298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.444298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.444298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.444662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.446017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:58:51.446017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:51.446978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:51.461715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:58:51.467550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.558466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 00:58:51.565783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.569306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:51.689385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161799d-a0e8-4d55-b7fa-62e084996efc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.696277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2548-c406-4c13-a4cf-daec079a842e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.725842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.726723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.727040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.727322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.735219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.735799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.736084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.736640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.737854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:51.739364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:51.740974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:51.746767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.748202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:51.749849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:51.763466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 00:58:51.769764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:58:51.770648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:58:51.772523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 00:58:51.772523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 00:58:51.772523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 00:58:51.772523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:58:51.772523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:58:51.777047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.838261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:51.863436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293c6a-7609-4a19-9a4e-cf866cd7d1c3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.886548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22166e7a-aba5-4ea3-bc03-fc8d57b1d596 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:51.901355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.902032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.902319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.902604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.903842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:51.905335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:51.906990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:58:51.912952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:51.927068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:51.927774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:51.928015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:51.928296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:51.929014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:58:51.929421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:51.930778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abce376-800f-4faa-bc9e-4541777714a3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:51.949971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:58:52.034532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:52.068291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504ef6c-86b9-4896-84a2-91dd31932ad8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:52.126824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:52.127828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:52.128261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:52.128753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:52.130588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.domain.id=default, user.id=52c6f46685584e8380771f008e5f1d77) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:58:52.131139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:52.133860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4277f-a409-4fa3-a3cb-4e5f7be18bbd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:52.146692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 00:58:51 2022] GET /identity/v3/role_assignments?user.id=52c6f46685584e8380771f008e5f1d77&scope.domain.id=default&role.id=132431a386d641a8aaeba0496b601812 => generated 233 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:58:55.290171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364de37-d1c2-4c11-bcc0-6539e19fea95 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.290714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364de37-d1c2-4c11-bcc0-6539e19fea95 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.290930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364de37-d1c2-4c11-bcc0-6539e19fea95 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.293720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:55.299210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.300161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.300651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.329849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:55.331504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:55.398292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ac692-eee7-45ad-9da3-58e471b30d35 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.398700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ac692-eee7-45ad-9da3-58e471b30d35 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.399406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ac692-eee7-45ad-9da3-58e471b30d35 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.401690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:55.408251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeb162-243a-4a0c-99a9-059c0af5a70e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.409016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeb162-243a-4a0c-99a9-059c0af5a70e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.409824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeb162-243a-4a0c-99a9-059c0af5a70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.440993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:58:55.460377 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-abaeb162-243a-4a0c-99a9-059c0af5a70e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:55.462619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-abaeb162-243a-4a0c-99a9-059c0af5a70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:55.491498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:55.636589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:58:55.637907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626725a6-c04d-41f6-9e6a-e02b14e9a3a9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.638266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626725a6-c04d-41f6-9e6a-e02b14e9a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.638588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626725a6-c04d-41f6-9e6a-e02b14e9a3a9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.640114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 14/26] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:55.640843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:55.660461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.660752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.661202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.684677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c45ca-4e76-4c06-b65b-4cdb37454a4f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:55.711991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:55.713259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:55.723225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7add74a-caf7-43f9-94c2-457b33e6121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:55.728888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7add74a-caf7-43f9-94c2-457b33e6121c None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.729434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7add74a-caf7-43f9-94c2-457b33e6121c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.729952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7add74a-caf7-43f9-94c2-457b33e6121c None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.731064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7add74a-caf7-43f9-94c2-457b33e6121c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:55.734312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7add74a-caf7-43f9-94c2-457b33e6121c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:55.838859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:55.840458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 15/27] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:58:55.847241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:55.909007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 13/28] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 00:58:55 2022] POST /identity/v3/roles => generated 225 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 00:58:55.910732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda17b56-0d19-44b0-9fe7-3679d18284ac None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:55.911073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda17b56-0d19-44b0-9fe7-3679d18284ac None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:55.911370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda17b56-0d19-44b0-9fe7-3679d18284ac None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:55.912769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 14/29] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:55.913577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda17b56-0d19-44b0-9fe7-3679d18284ac None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:56.124845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3356d-eb55-4323-8451-161fae1a2e08 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:56.174998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac2c2f-1ec7-4df3-8959-dbc874b9dbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:56.175838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac2c2f-1ec7-4df3-8959-dbc874b9dbaf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.176164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac2c2f-1ec7-4df3-8959-dbc874b9dbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.176510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac2c2f-1ec7-4df3-8959-dbc874b9dbaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.193564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac2c2f-1ec7-4df3-8959-dbc874b9dbaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f5ac2c2f-1ec7-4df3-8959-dbc874b9dbaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:58:56.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:58:56.201622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:58:56.202402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 16/30] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:58:56.203630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.203894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.204185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.225181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:56.226469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:56.232804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda17b56-0d19-44b0-9fe7-3679d18284ac None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:56.261616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:56.295703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:56.296952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.297583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.299464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.304348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:56.319253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:56.321852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 17/31] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:58:56.322758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:56.323012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 00:58:56.347429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:58:56.348679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:58:56.349975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 00:58:56.349975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 00:58:56.349975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 00:58:56.349975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:58:56.349975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 15/32] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:58:56.364183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:56.492759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc9d1-8b2c-4c16-8a2d-313e386f3192 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:56.543187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef185845-add6-45fc-b104-7854d88a95e8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:56.550659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:56.554371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.554658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.554987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.555759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:58:56.556073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:56.557732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:56.575326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 18/33] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:56.575326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:56.600239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:56.601167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.601634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.602057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.603620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:58:56.604097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:56.605720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209f4c0-67e0-436b-889c-d1d19b74d9c0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:56.634915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 16/34] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 00:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:56.634915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e175e-af18-472a-969b-2c4666ea3381 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.634915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e175e-af18-472a-969b-2c4666ea3381 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.634915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e175e-af18-472a-969b-2c4666ea3381 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.634915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 17/35] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:56.634915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e175e-af18-472a-969b-2c4666ea3381 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:56.648737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:56.708015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:56.757795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954dbe0-29e5-4dda-95bd-dcbc07ad3473 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:56.798014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8b661-1875-4bb0-a62a-e9867afac449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:56.798675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8b661-1875-4bb0-a62a-e9867afac449 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.798998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8b661-1875-4bb0-a62a-e9867afac449 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.799276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8b661-1875-4bb0-a62a-e9867afac449 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.800059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8b661-1875-4bb0-a62a-e9867afac449 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:56.801487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8b661-1875-4bb0-a62a-e9867afac449 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:56.802616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e175e-af18-472a-969b-2c4666ea3381 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:56.845509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:56.846208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.846503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.846785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.848180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:56.849710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:56.851701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:58:56 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:56.852215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:56.931783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 19/37] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 00:58:56 2022] POST /identity/v3/roles => generated 219 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:58:56.933531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf14021-7904-45ea-b384-6954208ac5f4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:56.934169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf14021-7904-45ea-b384-6954208ac5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:56.934169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf14021-7904-45ea-b384-6954208ac5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:56.961978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8bf14021-7904-45ea-b384-6954208ac5f4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:56.963836 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8bf14021-7904-45ea-b384-6954208ac5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:56.988647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b684430-fc01-4413-9eea-793040db0ea1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:57.011987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:57.044002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7f23c-7542-4227-9e51-a1dbc3945c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:57.044693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7f23c-7542-4227-9e51-a1dbc3945c4c None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.044974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7f23c-7542-4227-9e51-a1dbc3945c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.045250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7f23c-7542-4227-9e51-a1dbc3945c4c None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.046019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7f23c-7542-4227-9e51-a1dbc3945c4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:57.047710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7f23c-7542-4227-9e51-a1dbc3945c4c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:57.066483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 20/38] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:58:57.068517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc52bd-2c99-4ea6-b899-58c79916b846 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.069123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc52bd-2c99-4ea6-b899-58c79916b846 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.069695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc52bd-2c99-4ea6-b899-58c79916b846 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.071655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 21/39] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:57.072791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc52bd-2c99-4ea6-b899-58c79916b846 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:57.099210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 19/40] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 00:58:56 2022] POST /identity/v3/projects => generated 301 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:58:57.101204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.101616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.101996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.109529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:57.145878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:57.148751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:57.258080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 20/41] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:58:57.259050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:57.284263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc52bd-2c99-4ea6-b899-58c79916b846 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:57.329291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:57.331128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.331685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.332022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.333878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:57.336057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:57.345337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 22/42] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:58:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:57.345337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:57.403004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:57.411595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51872e91-cb2b-4732-b134-df67696558b5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:57.441563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:57.465226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2595d-4232-4f59-b86e-b42507512972 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:57.465521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:57.466128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.466376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.466685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.467504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:57.468995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:57.489419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 21/43] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 00:58:57 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:58:57.490276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:57.509752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:57.510817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.511254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.511693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.513598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:57.515150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:57.516926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 23/44] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:58:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:58:57.525088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:58:57.644343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be21004-5e45-460a-87e1-f8fbaae568d4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:57.683923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087b606-3b0f-4fad-b39e-1ecac92cfad2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:58:57.701812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4c4db-dc49-43c9-9ee0-1306bdb68aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:57.702512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4c4db-dc49-43c9-9ee0-1306bdb68aac None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.703299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4c4db-dc49-43c9-9ee0-1306bdb68aac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.703588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4c4db-dc49-43c9-9ee0-1306bdb68aac None admin] PATH_INFO: `/v3/domains/default/users/52c6f46685584e8380771f008e5f1d77/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.705767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4c4db-dc49-43c9-9ee0-1306bdb68aac None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=52c6f46685584e8380771f008e5f1d77, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:57.707353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4c4db-dc49-43c9-9ee0-1306bdb68aac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:57.726592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 22/45] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jul 28 00:58:57 2022] PUT /identity/v3/domains/default/users/52c6f46685584e8380771f008e5f1d77/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:58:57.753061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6b5a1-0c92-4eb5-961e-419b5578cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:58:57.754106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6b5a1-0c92-4eb5-961e-419b5578cd57 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:57.754399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6b5a1-0c92-4eb5-961e-419b5578cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:57.755224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6b5a1-0c92-4eb5-961e-419b5578cd57 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:57.755995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6b5a1-0c92-4eb5-961e-419b5578cd57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:58:57.757569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6b5a1-0c92-4eb5-961e-419b5578cd57 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:58:57.790965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 00:58:57 2022] POST /identity/v3/projects => generated 312 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:58:59.859651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495188e6-1648-4922-86c6-6907e7cde781 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:59.860184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495188e6-1648-4922-86c6-6907e7cde781 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:59.860618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495188e6-1648-4922-86c6-6907e7cde781 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:59.863435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:59.872515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:59.873081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:59.873081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:59.888471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:58:59.896404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:59.898426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:59.945157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a77bb-f4bd-4cec-b8d1-f463406626f0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:59.945446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a77bb-f4bd-4cec-b8d1-f463406626f0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:59.945736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a77bb-f4bd-4cec-b8d1-f463406626f0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:59.947225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 24/48] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:58:59.953095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:58:59.953387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:58:59.953683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:58:59.973518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:58:59.974782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:58:59.982879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 25/49] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:58:59.987412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:00.015921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.049162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 25/50] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:00.072633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.146883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6a16d-3901-4486-87c6-1306d018182c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.183963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcc7aa-9b2e-4a7d-b809-66c8e08290e9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.186244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.186993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.187454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.187564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.190724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:00.201889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:00.206996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:00.208114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:00.208939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 26/51] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:00.216234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.227621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.228326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.228613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.228899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.230167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:00.231710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:00.233362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 26/52] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:00.238990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.256107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:00.342866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379129d-1937-4b8d-bb6c-310c184a2ad4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.360636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096168f7-8d8f-45a0-97f4-333987faa2ad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.382665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.383408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.383738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.384058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.384923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:00.385432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:00.387113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:00.397466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 27/53] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:00.400748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.401467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.401753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.402032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.402740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:00.403757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.404240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:00.434738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 27/54] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 00:59:00 2022] POST /identity/v3/projects => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:00.471776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:00.522987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a49b9-e369-469c-a7ce-f1823368d81e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.565751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.566448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.566738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.567049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.568632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:00.570140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:00.571876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 28/55] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:00 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:00.578147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.651638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:00.690241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590cffc-dd5a-497f-a7b1-bb0ff074dd65 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.724371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290be93b-a101-4d70-8743-e2ca963419a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.725044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be93b-a101-4d70-8743-e2ca963419a3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.725361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be93b-a101-4d70-8743-e2ca963419a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.725661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be93b-a101-4d70-8743-e2ca963419a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.726876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290be93b-a101-4d70-8743-e2ca963419a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:00.728346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290be93b-a101-4d70-8743-e2ca963419a3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:00.730015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 28/56] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:00.736054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:00.838702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ea747-0c1c-4697-af96-bfcb8148651a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:00.864424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:00.873680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:00.874323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:00.874609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:00.874912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:00.875749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.domain.id=default, user.id=52c6f46685584e8380771f008e5f1d77) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:00.876060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:00.877565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd44d4-1027-4848-9b09-10c7a59da679 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:00.887977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 29/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jul 28 00:59:00 2022] GET /identity/v3/role_assignments?user.id=52c6f46685584e8380771f008e5f1d77&scope.domain.id=default&role.id=132431a386d641a8aaeba0496b601812 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:02.462110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143588d-9250-4964-8801-dea362593712 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:02.462829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143588d-9250-4964-8801-dea362593712 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:02.462829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143588d-9250-4964-8801-dea362593712 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:02.464124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:02.469962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:02.470499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:02.470499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:02.490240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:02.491583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:02.570809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:02.598238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143588d-9250-4964-8801-dea362593712 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:02.716554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143588d-9250-4964-8801-dea362593712 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:02.752683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8545800-9257-4359-a033-966ec19131d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:02.753336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8545800-9257-4359-a033-966ec19131d8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:02.753621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8545800-9257-4359-a033-966ec19131d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:02.753884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8545800-9257-4359-a033-966ec19131d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:02.755461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8545800-9257-4359-a033-966ec19131d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:02.756921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8545800-9257-4359-a033-966ec19131d8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:02.758543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:02.764989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:02.873051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522ab31-fab8-4b0d-b06c-d4d6edf8e0b3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:02.904642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:02.919570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3734080-774d-4473-8b43-0fcaaaa485d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:02.920208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3734080-774d-4473-8b43-0fcaaaa485d6 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:02.920504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3734080-774d-4473-8b43-0fcaaaa485d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:02.920782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3734080-774d-4473-8b43-0fcaaaa485d6 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:02.921489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3734080-774d-4473-8b43-0fcaaaa485d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:02.923008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3734080-774d-4473-8b43-0fcaaaa485d6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:02.952212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:02 2022] POST /identity/v3/users => generated 275 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:06.507555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2360a3-7834-4713-b04e-ad4088af534e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:06.509008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2360a3-7834-4713-b04e-ad4088af534e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:06.509652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2360a3-7834-4713-b04e-ad4088af534e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:06.512006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:06.518180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:06.519193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:06.519680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:06.548488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:06.550103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:06.652121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b69a6c-31af-4142-9368-0650c0960449 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:06.653071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b69a6c-31af-4142-9368-0650c0960449 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:06.653824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b69a6c-31af-4142-9368-0650c0960449 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:06.656600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 32/63] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:06.664739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 32/64] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:06.669047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d1c5-3ea7-45f6-b637-1cf2cb4ac2b2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:06.669877 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d1c5-3ea7-45f6-b637-1cf2cb4ac2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:06.670629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d1c5-3ea7-45f6-b637-1cf2cb4ac2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:06.691379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:06.724003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a751d1c5-3ea7-45f6-b637-1cf2cb4ac2b2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:06.729098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a751d1c5-3ea7-45f6-b637-1cf2cb4ac2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:06.738074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:06.949744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 33/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:06.972583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3cbff-7417-41d4-a1b2-89bd4fb7bd1c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:06.973014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3cbff-7417-41d4-a1b2-89bd4fb7bd1c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:06.973439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3cbff-7417-41d4-a1b2-89bd4fb7bd1c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:06.975978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 34/66] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:06.985020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:06.985660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:06.986218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:06.987847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0317a6-ae72-40ff-9449-a210b71d6569 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.003166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:07.012579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:07.014624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:07.052566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.054037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.054722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.055553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.077700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:07.086954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:07.089020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:07.090706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:07.092673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 33/67] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:07.093594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.155438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 35/68] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:07.157372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.273491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e7ce-266a-49a3-882a-b764afc1c1da None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.274731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:07.315145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.315913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.316265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.316605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.339843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:07.350606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484004b-1b46-4a5a-b749-255ce6f22106 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:07.351027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:07.351874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:07.352631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:07.353982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 34/69] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:07.355899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.398011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.398758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.399137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.399484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.400292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:07.400713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:07.402297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:07.414115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:07.414458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 36/70] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:07.416207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.547207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042c043-89b0-4bd0-aab3-9fec856a7ade None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.578089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ae325-98c3-452d-9251-d2c95acd5b09 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.598761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.599958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.600492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.601015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.603458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:07.617300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:07.618288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.619065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.619454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.619827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.620667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:07.621064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:07.622600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aab96-b3ca-4379-ab6b-b11a5835e960 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:07.625847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:07.626824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:07.627650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:07.628615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 35/71] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:07.628955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.633766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 37/72] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:07.635645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a951d4f-3df6-4dc7-9a68-3e6885827195 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.636077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a951d4f-3df6-4dc7-9a68-3e6885827195 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.636482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a951d4f-3df6-4dc7-9a68-3e6885827195 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.638038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 38/73] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:07.639353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb0642-52fa-4263-924f-140ecfe1031d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.639876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb0642-52fa-4263-924f-140ecfe1031d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.640607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb0642-52fa-4263-924f-140ecfe1031d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.642031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 39/74] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:07.644660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb0642-52fa-4263-924f-140ecfe1031d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.772368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:07.774646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb0642-52fa-4263-924f-140ecfe1031d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.795394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f7fd4-e094-4c45-a6d1-025aa7b2ee42 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:07.817369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.817929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.818218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.818499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.819289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:07.819601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:07.821057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:07.830219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 40/75] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:07.831129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:07.842713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ed96a-dd88-481f-8cc5-b76f86dc033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:07.843743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ed96a-dd88-481f-8cc5-b76f86dc033f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.844203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ed96a-dd88-481f-8cc5-b76f86dc033f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.844641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ed96a-dd88-481f-8cc5-b76f86dc033f None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.863083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ed96a-dd88-481f-8cc5-b76f86dc033f None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:07.864600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ed96a-dd88-481f-8cc5-b76f86dc033f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:07.866358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 36/76] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:07.867677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402fa8b-7d4c-4763-9dbe-48689caef7de None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.867978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402fa8b-7d4c-4763-9dbe-48689caef7de None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.868263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402fa8b-7d4c-4763-9dbe-48689caef7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.890432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f402fa8b-7d4c-4763-9dbe-48689caef7de None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:07.891741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f402fa8b-7d4c-4763-9dbe-48689caef7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:07.917818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:07.962290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 37/77] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:07.964104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:07.964484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:07.964847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:07.987171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:07.988937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:08.008565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af5b6d-be83-495f-ba62-09198196e49b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.068637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.069368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.069799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.070230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:08.071595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:08.073350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:08.075214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 41/78] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:07 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:08.076538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:08.099158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 38/79] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:08.100784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:08.106872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:08.205765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50f1e5-b343-496c-8dc4-6b09afe92bdf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.231539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:08.261809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35c181-865f-4ae4-94e1-9b8ea7763e78 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.264775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.265948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.266422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.266912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:08.269326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:08.272601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:08.288022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:08 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:08.289729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:08.337275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.338503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.339308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.339821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:08.448905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:08.451166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:08.451944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159615f-fdd7-4c91-9680-d541d28a3e6b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.456475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:08.515813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.516482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.516764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.517039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:08.526542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:08.528582 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:08.530282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 43/81] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:08.531113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:08.611919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 39/82] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:08.612648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:08.614564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:08.709769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96556fff-278f-48b6-9a88-c944694bd4ca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.757051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717d70-e51a-4b90-b05a-bea41247d57d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.761115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:08.783253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.784316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.784813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.785338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:08.801299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.801971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.802254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.802571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:08.814098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:08.819094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:08.820809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 40/83] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:08.822166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:08.876481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:08.878205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:08.936735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08984c7-8742-48b4-b645-553745a925dd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:08.961149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:08.987151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:08.987797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:08.988082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:08.988357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.022759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:09.063116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 44/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:09.063987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.086120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.087877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.213342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 41/85] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:09.214216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.249011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05b723-ae57-42c3-939c-8cec9597012c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:09.296242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:09.297317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:09.297830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:09.298354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.300071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.302554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.306192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:09.341946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 45/86] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:09 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:09.343055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.360459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc4c1-7d63-448c-b951-3958d0440973 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:09.375300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:09.416178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:09.417213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:09.417673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:09.418132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.420105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.421699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.423771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 42/87] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:09.425165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.529056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306474d-b597-40b6-a8a0-d7ef0244934d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:09.590698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:09.591682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:09.592237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:09.592696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.593980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:09.594487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.596732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.608151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318380f7-bda1-4591-ba30-00c3a515c37d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:09.611448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 46/88] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:09.615769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.662657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:09.670660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:09.671611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:09.671905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:09.672187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.673562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.675067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.677018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 43/89] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:09.678342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.736113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:09.789496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e9648-cbf5-40a4-ac85-90ed81cea871 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:09.850010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:09.851490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:09.851770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:09.852352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.853671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.855408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.855671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13613459-599b-41af-aadf-49db6f6839f4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:09.857175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:09.858056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:09.936721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:09.937676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:09.937970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:09.938254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:09.939327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] RBAC: Adding query filter params (role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=e7daa2b0f34a4966a673e1795e3cba54) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:09.939650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:09.941164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:09.948846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 44/91] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/role_assignments?user.id=e7daa2b0f34a4966a673e1795e3cba54&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:09.950620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:10.049017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec37a-4b5d-4607-9e3e-082f44c55436 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:10.104368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:10.105513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:10.105893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:10.106026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:10.106314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:10.112216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:10.113363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:10.123712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 48/92] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:10.124692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:10.141967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aeb65-86e7-40b0-b550-83202c6e8150 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:10.145899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:10.188804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:10.189458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:10.189740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:10.190015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:10.190829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=e7daa2b0f34a4966a673e1795e3cba54) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:10.192081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:10.203499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:10.214006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 45/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:09 2022] GET /identity/v3/role_assignments?user.id=e7daa2b0f34a4966a673e1795e3cba54&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:10.215088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:10.332885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038db890-43ef-4e88-b744-181e1fedfd14 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:10.379380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0bdc0-35f0-4e39-bb56-028cce31ea36 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:10.414260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:10.415216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:10.415658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:10.416073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:10.417189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=52c6f46685584e8380771f008e5f1d77) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:10.417719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:10.419528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:10.430597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:10.439705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:10.440371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:10.440663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:10.440947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:10.441971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 49/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:10 2022] GET /identity/v3/role_assignments?user.id=52c6f46685584e8380771f008e5f1d77&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:10.547424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:10.549040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:10.734400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 46/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:10.744034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:10.895361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:10.933791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaca08-7ca6-437c-915c-ca0f8f5df128 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:10.990709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bdef9b-095f-463d-a17a-92a486b0560f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:10.991320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdef9b-095f-463d-a17a-92a486b0560f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:10.991611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdef9b-095f-463d-a17a-92a486b0560f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:10.991895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdef9b-095f-463d-a17a-92a486b0560f None admin] PATH_INFO: `/v3/projects/f130dc0a92db49c88e128c55ff4fa7df` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:10.993202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdef9b-095f-463d-a17a-92a486b0560f None admin] RBAC: Authorizing `identity:get_project(project_id=f130dc0a92db49c88e128c55ff4fa7df)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:10.994777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdef9b-095f-463d-a17a-92a486b0560f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:10.996918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 50/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:10 2022] GET /identity/v3/projects/f130dc0a92db49c88e128c55ff4fa7df => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:11.003197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:11.074013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:11.176246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9bfa0-4038-4a17-a2e6-8ad6ad845b4f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:11.231600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:11.232285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:11.232605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:11.232950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:11.233788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=f130dc0a92db49c88e128c55ff4fa7df, user.id=e7daa2b0f34a4966a673e1795e3cba54) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:11.234111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:11.235819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f5602-d4c6-4a1d-9346-9b7f661622d6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:11.247836 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 47/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:11 2022] GET /identity/v3/role_assignments?user.id=e7daa2b0f34a4966a673e1795e3cba54&scope.project.id=f130dc0a92db49c88e128c55ff4fa7df&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:12.762700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b450dc-a961-4e17-961d-16993c6718b2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:12.762700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b450dc-a961-4e17-961d-16993c6718b2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:12.762700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b450dc-a961-4e17-961d-16993c6718b2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:12.762700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 51/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:12.769654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49b4b-92fd-4a98-a13f-56077c9b2a2f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:12.770655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49b4b-92fd-4a98-a13f-56077c9b2a2f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:12.770655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49b4b-92fd-4a98-a13f-56077c9b2a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:12.796254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4bc49b4b-92fd-4a98-a13f-56077c9b2a2f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:12.797515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4bc49b4b-92fd-4a98-a13f-56077c9b2a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:12.892074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 48/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:12.927016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b450dc-a961-4e17-961d-16993c6718b2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:13.074018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b450dc-a961-4e17-961d-16993c6718b2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:13.112180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:13.112862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.113146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.113419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.115532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d871aa8-7dcf-4d35-98bd-a41566d60f36 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.115936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d871aa8-7dcf-4d35-98bd-a41566d60f36 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.116180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d871aa8-7dcf-4d35-98bd-a41566d60f36 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.117486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 49/100] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:13.124183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.124592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.124825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.131501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:13.140615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:13.146070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:13.149318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:13.152819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:13.155144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:13.155144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:13.155653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 52/101] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:13.158879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:13.169724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:13.291297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d1de1-f8b7-4aa9-9af3-3a6ee8578129 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:13.304226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 50/102] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:13.342813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:13.343398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.343691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.343974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.344781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:13.345148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:13.346774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:13.356680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 53/103] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:13.362336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:13.368700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:13.436737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:13.482182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddd29a-e55b-4943-81f5-23e83bb9c0b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:13.520204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:13.520880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.521171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.521451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.523620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:13.525113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:13.527106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 54/104] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:13.535803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:13.653678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911fa27-6499-46b0-a368-b6f3327ce9ec None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:13.750498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:13.752335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.753495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.754547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] PATH_INFO: `/v3/users/b8f6af8654b6402face1518a8e832a99` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.762444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0b726-9f6d-4449-bb2f-10d161a4542f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:13.787790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] RBAC: Authorizing `identity:get_user(user_id=b8f6af8654b6402face1518a8e832a99)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:13.789720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:13.791164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 51/105] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:13 2022] GET /identity/v3/users/b8f6af8654b6402face1518a8e832a99 => generated 289 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:13.797107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:13.826706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:13.829188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:13.847684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa231e-e981-43de-8a9a-6c6a11b75fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:13.849308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa231e-e981-43de-8a9a-6c6a11b75fe2 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:13.850157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa231e-e981-43de-8a9a-6c6a11b75fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:13.850999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa231e-e981-43de-8a9a-6c6a11b75fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:13.852674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa231e-e981-43de-8a9a-6c6a11b75fe2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:13.856257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa231e-e981-43de-8a9a-6c6a11b75fe2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:13.908140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 55/106] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 00:59:13 2022] POST /identity/v3/projects => generated 302 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:14.012332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32b56f-71ef-47e1-be34-e76c2d52af44 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:14.058031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:14.058689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.059009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.059288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.060167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006a25e-a803-4930-929d-026d5049369d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.060658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006a25e-a803-4930-929d-026d5049369d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.061133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006a25e-a803-4930-929d-026d5049369d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.062769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 56/107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:14.068667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928366f6-38b5-44e8-be27-ea6cc02687ae None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.069188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928366f6-38b5-44e8-be27-ea6cc02687ae None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.069666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928366f6-38b5-44e8-be27-ea6cc02687ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.113576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-928366f6-38b5-44e8-be27-ea6cc02687ae None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:14.115434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-928366f6-38b5-44e8-be27-ea6cc02687ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:14.134194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:14.135941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:14.151265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:14.245322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 52/108] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:14.252324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:14.281612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 57/109] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:14.316407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa621990-71a5-4656-85bd-e0c501ea6c9b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.316719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa621990-71a5-4656-85bd-e0c501ea6c9b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.316989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa621990-71a5-4656-85bd-e0c501ea6c9b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.318409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 58/110] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:14.320048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa621990-71a5-4656-85bd-e0c501ea6c9b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:14.410265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb9d8d-ccc8-4ace-8d3f-c4481dea71d1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:14.431621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:14.477260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa621990-71a5-4656-85bd-e0c501ea6c9b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:14.487144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f70546-088f-4315-9436-dd8e68906421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:14.488189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70546-088f-4315-9436-dd8e68906421 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.488655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70546-088f-4315-9436-dd8e68906421 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.489211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70546-088f-4315-9436-dd8e68906421 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.492384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f70546-088f-4315-9436-dd8e68906421 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:14.495678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:14.499637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f70546-088f-4315-9436-dd8e68906421 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:14.512024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 53/111] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:14 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:14.512024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.512024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.512024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.540727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447fcf-62d7-451e-9e25-fbbda972b869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:14.541663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447fcf-62d7-451e-9e25-fbbda972b869 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.542050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447fcf-62d7-451e-9e25-fbbda972b869 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.542460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447fcf-62d7-451e-9e25-fbbda972b869 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.547971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:14.550549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:14.555461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447fcf-62d7-451e-9e25-fbbda972b869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4a447fcf-62d7-451e-9e25-fbbda972b869 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:14.560720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:14.562700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:14.564583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 59/112] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:14.566542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a283fdb-73cd-4af9-a17d-97c7a03f7287 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.571610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a283fdb-73cd-4af9-a17d-97c7a03f7287 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.572026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a283fdb-73cd-4af9-a17d-97c7a03f7287 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.578866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 60/113] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:14.578866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a283fdb-73cd-4af9-a17d-97c7a03f7287 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:14.690814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 54/114] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:14.692737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:14.700380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a283fdb-73cd-4af9-a17d-97c7a03f7287 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:14.751628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:14.752321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.752707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.753087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.754004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] RBAC: Adding query filter params (role.id=639f5d438fa44b96b0e715bcd0eab26a, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=b8f6af8654b6402face1518a8e832a99) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:14.754430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:14.756711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdf6f1-f86b-4fcd-b03e-390bc22c5931 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:14.766875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 61/115] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:14 2022] GET /identity/v3/role_assignments?user.id=b8f6af8654b6402face1518a8e832a99&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=639f5d438fa44b96b0e715bcd0eab26a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:14.766875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.766875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.766875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.786416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:14.790870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:14.820339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:14.868649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 62/116] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:14.869827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:14.922280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c557d69-2a39-4f44-8c9e-33293bd9cdad None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:14.933029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:14.989844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:14.991190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:14.992235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:14.992797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:14.993816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:14.994420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:14.996971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59e077-471b-48ce-a061-0e28da838506 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:15.011648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 55/117] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:15.013411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364d15-1e7d-4824-9b29-e8f6e8d0fddd None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.013950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364d15-1e7d-4824-9b29-e8f6e8d0fddd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.014455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364d15-1e7d-4824-9b29-e8f6e8d0fddd None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.014556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ab3bb-cf88-4b9a-a5e1-83bdba2a3bc0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.016212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 56/118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:15.017371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f364d15-1e7d-4824-9b29-e8f6e8d0fddd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.089042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.090010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.090296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.090577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.105793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:15.112743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:15.113604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:15.114338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 63/119] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:15.115456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.143107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f364d15-1e7d-4824-9b29-e8f6e8d0fddd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.172883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:15.186697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4292b-0291-4fb7-bd10-6bfb4c5b4957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.187747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4292b-0291-4fb7-bd10-6bfb4c5b4957 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.188188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4292b-0291-4fb7-bd10-6bfb4c5b4957 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.188608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4292b-0291-4fb7-bd10-6bfb4c5b4957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.203094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4292b-0291-4fb7-bd10-6bfb4c5b4957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-51c4292b-0291-4fb7-bd10-6bfb4c5b4957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:15.208992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:15.210055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:15.211020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:15.213022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 57/120] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:15.214613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.215171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.215646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.239299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:15.242312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:15.287990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:15.308855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2813c-2c8e-44a7-ab3a-246e79b7004b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.360820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.361764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.362136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.362419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.364368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:15.366710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:15.369240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 64/121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:15.375328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.393437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 58/122] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:15.394273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.546174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:15.554217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fc4a1-0aaf-448c-bea3-1d6758fef072 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.559000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8c9c3-833e-4e3b-b2ae-7e064aac92ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.594190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.595356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.595636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.596129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.596295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.596559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.596839 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.597137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.597570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:15.597883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:15.598464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:15.598994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:15.599298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:15.601012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:15.612645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 65/123] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:15.612645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.616496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 59/124] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:15.617654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.651719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:15.723513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ec064-1b3a-44f1-a1c4-4900d3548228 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.756209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42acb7-81ff-4cd0-8a97-92005a97fb1f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.763430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.764168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.764458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.764739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.765996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:15.767514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:15.769270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 66/125] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:15.774196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.797474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:15.808772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.809995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.810543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.811116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.824901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:15.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:15.832166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:15.832959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:15.835461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 60/126] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:15.836594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.883831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24ada8-5d9a-45e6-9506-9a57be3e40db None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:15.919290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:15.919529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:15.919811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:15.919980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:15.921144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:15.922600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:15.924246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 67/127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:15.928800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:15.945176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:15.980826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562184-7980-4a4e-9e9c-2bd043ce7e06 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.037510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.038979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.039661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.040199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.043579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.046751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.055069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 61/128] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:15 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:16.055069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.083024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:16.097855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8576bf-4413-47af-bae4-97fb4bfbddf3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.140854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.142218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.142712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.143322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05/users/e7daa2b0f34a4966a673e1795e3cba54/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.147345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] RBAC: Authorizing `identity:create_grant(project_id=5f2e064ab9374fbab081907969175e05, user_id=e7daa2b0f34a4966a673e1795e3cba54, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.149518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.165988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86485f9d-6a1f-4792-96f9-f02b92c47821 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.167141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 68/129] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:15 2022] PUT /identity/v3/projects/5f2e064ab9374fbab081907969175e05/users/e7daa2b0f34a4966a673e1795e3cba54/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:16.168200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.272511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.273232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.273521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.273846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.274600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:16.274933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.276499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.301262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 62/130] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:16.302263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.306576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e73e84-430c-4823-8537-5f0b3b52ef8a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.340398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:16.360370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.361297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.361777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.362217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.364462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.367014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.369584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 69/131] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:16 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:16.370507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.393925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:16.469204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056e4b-fb83-4b3a-92e6-ff18a3d5e4ea None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.509066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d5d58-8445-42b1-98b8-612b6dd044cf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.517564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d58f7d-9cca-48a2-8c06-5ae700703e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.518684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d58f7d-9cca-48a2-8c06-5ae700703e7e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.519098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d58f7d-9cca-48a2-8c06-5ae700703e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.519450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d58f7d-9cca-48a2-8c06-5ae700703e7e None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.520788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d58f7d-9cca-48a2-8c06-5ae700703e7e None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.522856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d58f7d-9cca-48a2-8c06-5ae700703e7e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.524644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 63/132] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:16 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:16.526030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356a8b8-343c-40fe-a631-d7fc3f1b88c5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.526329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356a8b8-343c-40fe-a631-d7fc3f1b88c5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.526653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356a8b8-343c-40fe-a631-d7fc3f1b88c5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.528354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 64/133] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:16.529054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356a8b8-343c-40fe-a631-d7fc3f1b88c5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.574271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa42b7f-e538-49bc-b7eb-53d4b62c2f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.575404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa42b7f-e538-49bc-b7eb-53d4b62c2f97 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.575938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa42b7f-e538-49bc-b7eb-53d4b62c2f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.576472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa42b7f-e538-49bc-b7eb-53d4b62c2f97 None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.579611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa42b7f-e538-49bc-b7eb-53d4b62c2f97 None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.582754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa42b7f-e538-49bc-b7eb-53d4b62c2f97 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.587248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 70/134] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:16 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:16.593242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57308a84-befa-4659-9911-026d787fed1c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.593785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57308a84-befa-4659-9911-026d787fed1c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.594258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57308a84-befa-4659-9911-026d787fed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.622527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-57308a84-befa-4659-9911-026d787fed1c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:16.623843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-57308a84-befa-4659-9911-026d787fed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:16.640305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:16.664518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356a8b8-343c-40fe-a631-d7fc3f1b88c5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.701894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 71/135] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:16.702337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.702782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57308a84-befa-4659-9911-026d787fed1c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.708909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.708909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.708909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05/users/e7daa2b0f34a4966a673e1795e3cba54/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.715719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f2e064ab9374fbab081907969175e05, user_id=e7daa2b0f34a4966a673e1795e3cba54, role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.717740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.725737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:16.756961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 65/136] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:16 2022] PUT /identity/v3/projects/5f2e064ab9374fbab081907969175e05/users/e7daa2b0f34a4966a673e1795e3cba54/roles/8fc2a014f1e24e08a63b265436f41eec => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:16.757959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.851229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57308a84-befa-4659-9911-026d787fed1c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:16.933588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:16.934389 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:16.934596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:16.934904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05/users/52c6f46685584e8380771f008e5f1d77/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:16.940548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] RBAC: Authorizing `identity:create_grant(project_id=5f2e064ab9374fbab081907969175e05, user_id=52c6f46685584e8380771f008e5f1d77, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:16.942305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:16.966089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 72/137] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:16 2022] PUT /identity/v3/projects/5f2e064ab9374fbab081907969175e05/users/52c6f46685584e8380771f008e5f1d77/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:16.966760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:16.982375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962994c7-2e5a-42d8-bfa3-62c047f70404 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:17.043280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:17.043975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.044259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.044538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] PATH_INFO: `/v3/projects/f130dc0a92db49c88e128c55ff4fa7df` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.047255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] RBAC: Authorizing `identity:get_project(project_id=f130dc0a92db49c88e128c55ff4fa7df)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:17.048786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:17.050829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 66/138] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:16 2022] GET /identity/v3/projects/f130dc0a92db49c88e128c55ff4fa7df => generated 312 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:17.057814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:17.085452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:17.167717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:17.247586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67962ad-7523-42ee-a749-6f072f6b81aa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:17.269516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eff3fc-5bbe-4aae-8bae-2993b9235e78 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:17.317935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:17.318935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.319359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.319770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.321139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:17.322862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:17.324668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 73/139] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:17.326708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec3e3-655a-47a9-aaff-75aea6086d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:17.327165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec3e3-655a-47a9-aaff-75aea6086d6c None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.327689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec3e3-655a-47a9-aaff-75aea6086d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.328163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec3e3-655a-47a9-aaff-75aea6086d6c None admin] PATH_INFO: `/v3/projects/f130dc0a92db49c88e128c55ff4fa7df/users/e7daa2b0f34a4966a673e1795e3cba54/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.330921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:17.337356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec3e3-655a-47a9-aaff-75aea6086d6c None admin] RBAC: Authorizing `identity:create_grant(project_id=f130dc0a92db49c88e128c55ff4fa7df, user_id=e7daa2b0f34a4966a673e1795e3cba54, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:17.339431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec3e3-655a-47a9-aaff-75aea6086d6c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:17.377195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 67/140] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:17 2022] PUT /identity/v3/projects/f130dc0a92db49c88e128c55ff4fa7df/users/e7daa2b0f34a4966a673e1795e3cba54/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:17.378610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6f1a-96f5-48c2-8321-bd23bec6571d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.379064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6f1a-96f5-48c2-8321-bd23bec6571d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.379350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6f1a-96f5-48c2-8321-bd23bec6571d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.381005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 68/141] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:17.393189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.393498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.393756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.436202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:17.437218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:17.469630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850d0e2-fc09-4281-b79a-4ff1794b700f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:17.515622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:17.516771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.517352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.517820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.518737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:17.520483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:17.542449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:17.551641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 74/142] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:17 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 00:59:17.633229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:17.646731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 69/143] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:17.726207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:17.847860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d9f75-3631-4c2f-90f1-7d77df4f1d66 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:17.889307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:17.890354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:17.890865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:17.891361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:17.904597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:17.910776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:17.911720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:17.912474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:17.915584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 75/144] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:17.928091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:18.111099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d99707-7d80-4a47-b0d8-a5a1d5e11098 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:18.158568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:18.159657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:18.160143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:18.160613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:18.161671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:18.162202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:18.163942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:18.177323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 70/145] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:18.183147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:18.265253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:18.315864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0adc-f715-4a87-915c-45e93abb7ecf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:18.361596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:18.362185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:18.362468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:18.362743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] PATH_INFO: `/v3/users/b8f6af8654b6402face1518a8e832a99` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:18.364621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] RBAC: Authorizing `identity:get_user(user_id=b8f6af8654b6402face1518a8e832a99)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:18.366860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:18.368944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 76/146] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:18 2022] GET /identity/v3/users/b8f6af8654b6402face1518a8e832a99 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:18.378109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:18.501475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:18.550259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352e00-dddc-467f-b902-33c7d72b29e6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:18.598634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e862a09-9076-4008-90f7-15f06d10a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:18.599745 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e862a09-9076-4008-90f7-15f06d10a77b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:18.600230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e862a09-9076-4008-90f7-15f06d10a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:18.600728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e862a09-9076-4008-90f7-15f06d10a77b None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:18.602232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e862a09-9076-4008-90f7-15f06d10a77b None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:18.604053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e862a09-9076-4008-90f7-15f06d10a77b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:18.606099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 71/147] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:18 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:18.616077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:18.859842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb828b-93a8-4455-8a64-ed74da112d5a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:18.912731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:18.926813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3dbe0-0f56-4e1d-9108-842ecfb3cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:18.928278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3dbe0-0f56-4e1d-9108-842ecfb3cd00 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:18.928943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3dbe0-0f56-4e1d-9108-842ecfb3cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:18.929704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3dbe0-0f56-4e1d-9108-842ecfb3cd00 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05/users/b8f6af8654b6402face1518a8e832a99/roles/639f5d438fa44b96b0e715bcd0eab26a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:18.933466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3dbe0-0f56-4e1d-9108-842ecfb3cd00 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f2e064ab9374fbab081907969175e05, user_id=b8f6af8654b6402face1518a8e832a99, role_id=639f5d438fa44b96b0e715bcd0eab26a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:18.936477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3dbe0-0f56-4e1d-9108-842ecfb3cd00 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:18.962126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 77/148] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:18 2022] PUT /identity/v3/projects/5f2e064ab9374fbab081907969175e05/users/b8f6af8654b6402face1518a8e832a99/roles/639f5d438fa44b96b0e715bcd0eab26a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:19.091068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2d52f-183a-4d10-b777-c83ca947bc0f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.091367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2d52f-183a-4d10-b777-c83ca947bc0f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.091711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2d52f-183a-4d10-b777-c83ca947bc0f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.093098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 72/149] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:19.099994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b7e9b-1210-4a24-978b-cd020cf6433a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.100927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b7e9b-1210-4a24-978b-cd020cf6433a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.101693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b7e9b-1210-4a24-978b-cd020cf6433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.143075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-440b7e9b-1210-4a24-978b-cd020cf6433a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:19.145647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-440b7e9b-1210-4a24-978b-cd020cf6433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:19.295364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 78/150] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:19.316920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d317c-3985-4963-8159-d03ae4e437f3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.317731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d317c-3985-4963-8159-d03ae4e437f3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.317731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d317c-3985-4963-8159-d03ae4e437f3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.318875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 73/151] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:19.325654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.326534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.327397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.334681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d317c-3985-4963-8159-d03ae4e437f3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:19.368021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:19.370718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:19.492807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d317c-3985-4963-8159-d03ae4e437f3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:19.506484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 79/152] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:19.531238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:19.546638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:19.547565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.547992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.548509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.552768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:19.568272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:19.573956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:19.574963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:19.575838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:19.577951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 74/153] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:19.583979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:19.617646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:19.715729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002429b7-43c4-4d94-9304-e06924eeb16f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:19.735254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cf4c-1e52-4a1f-8e47-3807407784d2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:19.779657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:19.780354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.780662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.780899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.791582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:19.792454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:19.792868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:19.793241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:19.794271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:19.794665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:19.796109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:19.810913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:19.812817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:19.813758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 80/154] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:19.820119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:19.828763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 75/155] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:19.835716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:20.034789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:20.085114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483208c-57f0-49db-92f4-113115a3420b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:20.105450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:20.202556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e381d-42ef-4af9-a69f-d127e91d918a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:20.219617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:20.220376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:20.220556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:20.220832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:20.221580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:20.221887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:20.223329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:20.237497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 81/156] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:20.243255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:20.305828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:20.307010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:20.307604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:20.308214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:20.310396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:20.313140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:20.316402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 76/157] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:19 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:20.324581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:20.464756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e38450-7e4f-4633-868f-3f70cc61dbdb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:20.522521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:20.523217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:20.523497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:20.523769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:20.526039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:20.527538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:20.529188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 82/158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:20 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:20.534774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:20.614672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:20.617344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7b52-61a8-4d96-bee8-4cc5c4c59556 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:20.641384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:20.680564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:20.681604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:20.682163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:20.682645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:20.690879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c6ec6-0647-4ec1-bd30-a79100e14b6f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:20.751868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7ac6f-27ef-40b5-9d66-0cc744cf874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:20.752441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7ac6f-27ef-40b5-9d66-0cc744cf874d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:20.752725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7ac6f-27ef-40b5-9d66-0cc744cf874d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:20.753480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7ac6f-27ef-40b5-9d66-0cc744cf874d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:20.788526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:20.790425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:20.867046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7ac6f-27ef-40b5-9d66-0cc744cf874d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:20.868420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7ac6f-27ef-40b5-9d66-0cc744cf874d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:20.941690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:20.942221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 77/159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:20.944235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:20.976135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:21.045602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 83/160] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 00:59:21.050865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf84c58-8450-4c9a-b260-ae5e00650ac4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.051468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf84c58-8450-4c9a-b260-ae5e00650ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.051890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf84c58-8450-4c9a-b260-ae5e00650ac4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.053457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 84/161] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:21.060058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf84c58-8450-4c9a-b260-ae5e00650ac4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:21.215683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf84c58-8450-4c9a-b260-ae5e00650ac4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:21.245352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36fcc-58c3-4618-9b9f-2f05110b9c5d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:21.279721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88abdb-8b46-499e-8778-5b6c1c933b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:21.280666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88abdb-8b46-499e-8778-5b6c1c933b94 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.281117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88abdb-8b46-499e-8778-5b6c1c933b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.281526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88abdb-8b46-499e-8778-5b6c1c933b94 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.283068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88abdb-8b46-499e-8778-5b6c1c933b94 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:21.284717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88abdb-8b46-499e-8778-5b6c1c933b94 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:21.286951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 85/162] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:21 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:21.288402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.288836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.289239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.315998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:21.324525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:21.325899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:21.332379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:21.333076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.333372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.333742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.335123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:21.336606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:21.338672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 78/163] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:20 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:21.339563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:21.429373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 86/164] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:21.430293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:21.589229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a6a7c-d7bc-492a-b3fd-0eff18752b56 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:21.614799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:21.647636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:21.648299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.648581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.648854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.649674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=e7daa2b0f34a4966a673e1795e3cba54) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:21.649986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:21.651576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:21.655232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:21.657260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2cfd1-3261-4a70-9b05-c2f87a871987 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:21.658249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 87/165] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:21 2022] GET /identity/v3/role_assignments?user.id=e7daa2b0f34a4966a673e1795e3cba54&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:21.663189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:21.776127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:21.777107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.777401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.777679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.778521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] RBAC: Adding query filter params (role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=e7daa2b0f34a4966a673e1795e3cba54) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:21.778865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:21.780402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:21.797021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 79/166] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:21 2022] GET /identity/v3/role_assignments?user.id=e7daa2b0f34a4966a673e1795e3cba54&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 608 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:21.827049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a13e5-a632-40fe-ad07-50b7c5fb24e8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:21.875005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:21.875597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:21.875879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:21.876152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:21.901295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:21.906973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:21.907942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:21.910043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 88/167] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:21.918014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:22.184146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3544f-fd1a-41ce-b607-003ba2a021da None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:22.275726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:22.276405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:22.276870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:22.277163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:22.277928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:22.278244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:22.279686 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:22.290005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 80/168] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:22.295723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:22.425572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:22.557019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f3b72-ab31-4ac8-a1c9-47c369c17dcf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:22.631020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5267c2-83da-4ce6-a8f2-9c1b526f79c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:22.631805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5267c2-83da-4ce6-a8f2-9c1b526f79c1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:22.632045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5267c2-83da-4ce6-a8f2-9c1b526f79c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:22.632412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5267c2-83da-4ce6-a8f2-9c1b526f79c1 None admin] PATH_INFO: `/v3/users/52c6f46685584e8380771f008e5f1d77` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:22.638187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5267c2-83da-4ce6-a8f2-9c1b526f79c1 None admin] RBAC: Authorizing `identity:get_user(user_id=52c6f46685584e8380771f008e5f1d77)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:22.639900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5267c2-83da-4ce6-a8f2-9c1b526f79c1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:22.641797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 89/169] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:22 2022] GET /identity/v3/users/52c6f46685584e8380771f008e5f1d77 => generated 247 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:22.647766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:22.675312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:22.807403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291b26-f910-4cef-8088-80244cb46811 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:22.839566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef19af-5545-462d-ad27-8ef8622285b2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:22.839898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef19af-5545-462d-ad27-8ef8622285b2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:22.840155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef19af-5545-462d-ad27-8ef8622285b2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:22.842257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 90/170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:22.863907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:22.864553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:22.864841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:22.865122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:22.869731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766eee3-f08d-43c7-b3fb-6ea646b3ec2c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:22.870053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766eee3-f08d-43c7-b3fb-6ea646b3ec2c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:22.870457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766eee3-f08d-43c7-b3fb-6ea646b3ec2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:22.904516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a766eee3-f08d-43c7-b3fb-6ea646b3ec2c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:22.905742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a766eee3-f08d-43c7-b3fb-6ea646b3ec2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:22.964380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:22.965890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:22.983359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.029038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 91/171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:23.030133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec636f0-d849-4eca-abfb-504d7fd158ce None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.030421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec636f0-d849-4eca-abfb-504d7fd158ce None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.030696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec636f0-d849-4eca-abfb-504d7fd158ce None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.032380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 92/172] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:23.048347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.048685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.048901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.092678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:23.093941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:23.108754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 81/173] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:23.109650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.179943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.208723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 93/174] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:23.209598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.288415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c6331-9522-442a-ad8d-c6475cddee8f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:23.315551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.335290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:23.336470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.337238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.337977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.353306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:23.359514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:23.360586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:23.361513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:23.364150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 82/175] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:23.369962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.394787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815df189-ee0f-413b-9ba4-b48db7029be0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:23.433583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19203d7f-b9b6-422c-b1dd-b5df4e899bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:23.434249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203d7f-b9b6-422c-b1dd-b5df4e899bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.434534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203d7f-b9b6-422c-b1dd-b5df4e899bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.434812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203d7f-b9b6-422c-b1dd-b5df4e899bb6 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.436031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19203d7f-b9b6-422c-b1dd-b5df4e899bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:23.437488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19203d7f-b9b6-422c-b1dd-b5df4e899bb6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:23.439214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 94/176] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:23.444293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55f77-2b21-4274-afaf-f9b90b515c2b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.444558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55f77-2b21-4274-afaf-f9b90b515c2b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.444839 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55f77-2b21-4274-afaf-f9b90b515c2b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.446173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 95/177] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:23.447206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55f77-2b21-4274-afaf-f9b90b515c2b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.498773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111070-ec06-4076-bd30-7aa22c8c0821 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:23.510141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.535122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99958bbc-cf63-467a-b837-10b0ec5a7939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:23.535784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958bbc-cf63-467a-b837-10b0ec5a7939 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.536069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958bbc-cf63-467a-b837-10b0ec5a7939 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.536355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958bbc-cf63-467a-b837-10b0ec5a7939 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.547513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99958bbc-cf63-467a-b837-10b0ec5a7939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-99958bbc-cf63-467a-b837-10b0ec5a7939 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:23.552521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:23.553509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:23.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 83/178] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:23.555382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55f77-2b21-4274-afaf-f9b90b515c2b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:23.555456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038d44b-f0a8-420d-afd6-addba079ee7c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.555456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038d44b-f0a8-420d-afd6-addba079ee7c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.555579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038d44b-f0a8-420d-afd6-addba079ee7c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.556854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 84/179] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:23.561942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a518c0-a4a7-4b86-9ade-78c52caecaad None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.561942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a518c0-a4a7-4b86-9ade-78c52caecaad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.561942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a518c0-a4a7-4b86-9ade-78c52caecaad None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.561942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 85/180] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:23.561942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a518c0-a4a7-4b86-9ade-78c52caecaad None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.594233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.597857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:23.598511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.598825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.599107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.599843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:23.600150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:23.601574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2871a-5084-402f-935b-66290ec16d20 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:23.612202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 96/181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:23.613438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.613735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.614274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.635728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:23.636970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:23.674391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a518c0-a4a7-4b86-9ade-78c52caecaad None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:23.712406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:23.713128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.713411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.713688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.714509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=52c6f46685584e8380771f008e5f1d77) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:23.714652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 97/182] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:23.715525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.719117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:23.720805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba893-ef26-4853-b551-8ee0a96aaade None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:23.727599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 86/183] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/role_assignments?user.id=52c6f46685584e8380771f008e5f1d77&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:23.729729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801df567-9fab-465c-acdd-cededd52c717 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.729729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801df567-9fab-465c-acdd-cededd52c717 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.729729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801df567-9fab-465c-acdd-cededd52c717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.754021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-801df567-9fab-465c-acdd-cededd52c717 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:23.755855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-801df567-9fab-465c-acdd-cededd52c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:23.789179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.866961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 87/184] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:23.867152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70daaa-4b8d-42a7-ba5e-5b868243bc07 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:23.867240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.867240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.867240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.878461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:23.880305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:23.904198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:23.919362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:23.920012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:23.920300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:23.920576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:23.921693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:23.921693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:23.923095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:23.934421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 98/185] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:23.935607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:23.980935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 88/186] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:23.981921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.052464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b6fa1-7956-4719-a7ac-a77e8f96c755 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.063661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.092103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.092344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cadd2-26a8-411c-a7aa-69faa952890e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.092744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.093028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.094220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] PATH_INFO: `/v3/users/e8708f9d98754ff88ae858e35d39eb29` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.111612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] RBAC: Authorizing `identity:get_user(user_id=e8708f9d98754ff88ae858e35d39eb29)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.113104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.114891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 99/187] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/users/e8708f9d98754ff88ae858e35d39eb29 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:24.115715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.133734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.134318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.134692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.135010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.147207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:24.152266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:24.153332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 89/188] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:24.159491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.173770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.235107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98928a6-f2b6-4412-953d-60dd6d0fe241 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.272338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ff60-a5b0-46b4-84d6-d8b70f74f467 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.273362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.274036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.274341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.274591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.275918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.277411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.279108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 100/189] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:24.279897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.316325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.316325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.316325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.316325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] PATH_INFO: `/v3/users/e7daa2b0f34a4966a673e1795e3cba54` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.320422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] RBAC: Authorizing `identity:get_user(user_id=e7daa2b0f34a4966a673e1795e3cba54)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.320422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.321840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 90/190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/users/e7daa2b0f34a4966a673e1795e3cba54 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:24.323891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.324792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.396182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae118-13a9-45dc-a003-44c1f9199966 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.420146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.433903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.434707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.435019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.435301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.436668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1d88-4dce-4839-8823-10b2a215d406 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.449390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:24.454948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:24.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 101/191] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:24.457176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.481634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.482284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.482570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.482904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.559120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.561004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.578713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.586063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1a3d-2b7f-449b-9f81-97b6debd3f29 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.633201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.633838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.634117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.634392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.635200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:24.635506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.636932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.645918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 102/192] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:24.646823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.652691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.673330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 91/193] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:24.674512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.777875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e6633-defc-42a4-9211-3d4e088c0485 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.790273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e0739-8de9-4bf4-804d-1d5a15256084 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:24.813846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.814493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.814775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.815073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.819418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.819418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.836167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:24.836809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:24.837120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:24.837388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:24.842072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 103/194] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:24 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:24.842072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:24.870700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.916516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:24.940734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:24.942574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:24.984109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1f8c5-4743-4f11-b6bb-4367ce52d749 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.028208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.028864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.029153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.031644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.031644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:25.031644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.031987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.042075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 104/195] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:25.042614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.078579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 92/196] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:25.080358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.160893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:25.191773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777725cc-a930-4646-9a87-9d3699c66843 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.211534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:25.242345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea7a38-c512-40f7-bda4-c96cb659dc76 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.247193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.247871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.248319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.248537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] PATH_INFO: `/v3/users/b8f6af8654b6402face1518a8e832a99` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.255836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] RBAC: Authorizing `identity:get_user(user_id=b8f6af8654b6402face1518a8e832a99)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.259344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.262381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 105/197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/users/b8f6af8654b6402face1518a8e832a99 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:25.264173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.297218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.298087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.298388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.298693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.300052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.301757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.303689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 93/198] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:25.309123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.391197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e8d36-c1c0-4d5e-a1b5-a5bd517a7485 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.436369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e042f-35fe-49ae-99c1-ea94b7b09d64 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.439903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.440586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.440867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.441204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] PATH_INFO: `/v3/users/e8708f9d98754ff88ae858e35d39eb29` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.442575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] RBAC: Authorizing `identity:get_user(user_id=e8708f9d98754ff88ae858e35d39eb29)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.444011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.446572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 106/199] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/users/e8708f9d98754ff88ae858e35d39eb29 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:25.449999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.471056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:25.480482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.481503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.481933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.482354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] PATH_INFO: `/v3/projects/f130dc0a92db49c88e128c55ff4fa7df` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.483939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] RBAC: Authorizing `identity:get_project(project_id=f130dc0a92db49c88e128c55ff4fa7df)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.485769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.488127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 94/200] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/projects/f130dc0a92db49c88e128c55ff4fa7df => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:25.489329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.520932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:25.603189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e5a7-2ba9-41c3-bb94-8d0903718180 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.641767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e211e-2300-4d83-b2de-157434e26a5f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.654901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.656206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.656670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.657122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.684103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.684752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:25.685054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:25.685310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:25.686122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=e8708f9d98754ff88ae858e35d39eb29) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:25.686612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.688361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.703047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 95/201] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/role_assignments?user.id=e8708f9d98754ff88ae858e35d39eb29&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:25.703047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.775818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:25.777938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:25.795316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:25.914561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:25.932268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfdd8c-f47f-438c-9b0d-ddc87d86525b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:25.953937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 107/202] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:25.954752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:25.998782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:25.999865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:26.000346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:26.000806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:26.073656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f178c1f-8b8e-4b9b-a0ab-eceabf41a854 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:26.096320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:26.110969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:26.112617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:26.112854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:26.113196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:26.114119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=f130dc0a92db49c88e128c55ff4fa7df, user.id=e7daa2b0f34a4966a673e1795e3cba54) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:26.114454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:26.116042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:26.122718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 108/203] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/role_assignments?user.id=e7daa2b0f34a4966a673e1795e3cba54&scope.project.id=f130dc0a92db49c88e128c55ff4fa7df&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:26.123488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:26.208265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:26.209680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:26.235803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce86893-0028-40f0-996c-5281416f2bfd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:26.276515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:26.277776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:26.278248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:26.278639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:26.280056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:26.282330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:26.284466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 109/204] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:26 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:26.292786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:26.381478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:26.433934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:26.461960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07801-cb97-4e62-a08f-fb5478f94d65 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:26.470198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 96/205] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:26.478810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:26.516602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:26.517689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:26.518343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:26.518806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:26.519843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] RBAC: Adding query filter params (role.id=639f5d438fa44b96b0e715bcd0eab26a, scope.project.id=5f2e064ab9374fbab081907969175e05, user.id=b8f6af8654b6402face1518a8e832a99) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:26.520346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:26.522197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:26.529272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 110/206] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:26 2022] GET /identity/v3/role_assignments?user.id=b8f6af8654b6402face1518a8e832a99&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=639f5d438fa44b96b0e715bcd0eab26a => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:26.625232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbde7e-793f-48cb-915d-27da70e5de7b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:26.680824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8f8ae-5899-44cf-9e97-619bae462f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:26.681513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8f8ae-5899-44cf-9e97-619bae462f73 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:26.681927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8f8ae-5899-44cf-9e97-619bae462f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:26.682201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8f8ae-5899-44cf-9e97-619bae462f73 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:26.683443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8f8ae-5899-44cf-9e97-619bae462f73 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:26.684917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8f8ae-5899-44cf-9e97-619bae462f73 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:26.686614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 97/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:26 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:26.693285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:26.873697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d301-2c6d-42e8-b5c3-9ade768a33c8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:26.887237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:26.913616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:26.914680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:26.915242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:26.915755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:26.917177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] RBAC: Adding query filter params (role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=e8708f9d98754ff88ae858e35d39eb29) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:26.917561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:26.919034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d94fd-a93b-4896-9b3c-82e991782909 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:26.925599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 111/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:26 2022] GET /identity/v3/role_assignments?user.id=e8708f9d98754ff88ae858e35d39eb29&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:27.300136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c41a8b-40f5-4ca5-8a4d-361582e42097 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.300655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c41a8b-40f5-4ca5-8a4d-361582e42097 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.301052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c41a8b-40f5-4ca5-8a4d-361582e42097 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.302778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 98/209] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:27.308302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.308713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.309091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.328404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:27.329619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:27.333262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3bad0-635b-44e1-9c67-1dfe07995697 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.333553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3bad0-635b-44e1-9c67-1dfe07995697 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.333856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3bad0-635b-44e1-9c67-1dfe07995697 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.336464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 99/210] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:27.342071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.342364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.342650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.366428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:27.367673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:27.403400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 112/211] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:27.429802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:27.443805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:27.476678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 100/212] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:27.495560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:27.571956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70b294-da65-4e90-94db-a71c88ff4c17 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:27.574863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:27.619513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:27.620567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.620792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d965c-705c-4726-ab1e-ae94452d8866 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:27.621274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.621742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.624486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:27.626222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:27.633377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 113/213] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:27.636315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:27.665630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:27.666359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.666748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.667078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.682288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:27.688615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:27.689982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:27.690762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 101/214] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:27.698285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:27.756757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:27.782114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678b4d-3a4d-4d6e-892b-2441231de986 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:27.827049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:27.827387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6c98-c7e8-40bb-a834-225e9d570042 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:27.828239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.828963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.829433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.831285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:27.833070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:27.867489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 114/215] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:27 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:27.888312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:27.888979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:27.889268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:27.889546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:27.890298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:27.890606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:27.892050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2eb02-f07a-4179-966a-51533aaa1a5e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:27.902580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 102/216] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:27.912271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:27.947372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:28.013985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b331eb-6266-4744-933e-96851bdfa8c7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.014262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b331eb-6266-4744-933e-96851bdfa8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.014535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b331eb-6266-4744-933e-96851bdfa8c7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.015967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 103/217] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:28.021680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.022037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.022386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.042444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:28.044121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:28.054331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df1116-7053-4289-8cdf-009a166f6137 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:28.103661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:28.104290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.104525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.104822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] PATH_INFO: `/v3/users/71c019b9826c4539b97215ac8535a1ae` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.120427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] RBAC: Authorizing `identity:get_user(user_id=71c019b9826c4539b97215ac8535a1ae)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:28.121969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:28.123846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 115/218] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:27 2022] GET /identity/v3/users/71c019b9826c4539b97215ac8535a1ae => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:28.126630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 104/219] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:28.129575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.150035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.186976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:28.235457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:28.261968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1efb0b-d60a-40e3-861a-bf26bdefe183 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:28.283155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101a319-acf7-4a2d-b5bc-05de1380dcd9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:28.312657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:28.313627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.314119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.314605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.334809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:28.335743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.336231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.336604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.349016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:28.354855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:28.355948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:28.356916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:28.363063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 105/220] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:28.365079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.412991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:28.415301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:28.493464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:28.515882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd14ae-5377-454d-bd72-aa74e302b9e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:28.536867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:28.559856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 116/221] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:28.567056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.578170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:28.578825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.579205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.579424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.580177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:28.580494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:28.581921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:28.599369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 106/222] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:28.607544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.760947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d48a2-fed9-4a2d-bf41-6ae6d370b560 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:28.799941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbd2b4-8a38-431b-ba17-86398f2c3d7d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:28.810936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:28.811614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.811898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.812183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.815892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:28.817472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:28.819371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 117/223] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:28.837647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.844117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:28.844784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:28.845066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:28.845345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] PATH_INFO: `/v3/users/e8708f9d98754ff88ae858e35d39eb29` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:28.846538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] RBAC: Authorizing `identity:get_user(user_id=e8708f9d98754ff88ae858e35d39eb29)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:28.848020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:28.850043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 107/224] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/users/e8708f9d98754ff88ae858e35d39eb29 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:28.855960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:28.906221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:28.959231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:29.042866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff4fe9-b5e0-43a1-9195-31455387a895 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:29.079877 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:29.080524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.080805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.081080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.081909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=71c019b9826c4539b97215ac8535a1ae) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:29.082218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:29.083821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389bcc4-5b87-4864-8aee-d99bba30d50e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:29.090264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 118/225] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/role_assignments?user.id=71c019b9826c4539b97215ac8535a1ae&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:29.097350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b157133-b6da-4330-9b86-26f854c1d976 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:29.134569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28c17-e37e-4d98-9362-7883db1ed5ce None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.135122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28c17-e37e-4d98-9362-7883db1ed5ce None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.135406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28c17-e37e-4d98-9362-7883db1ed5ce None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.137345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 119/226] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:29.139828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:29.140572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.140860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.141136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.142639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:29.144347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:29.147211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 108/227] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:28 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:29.148234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.148523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.148798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.156925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:29.175438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:29.176689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:29.262636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 120/228] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:29.309262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:29.367458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:29.382971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:29.404758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f85419-f5c4-4299-b36a-b9bb5e74961c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:29.449861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e9ea3-edce-44b5-9128-c731f42fa290 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:29.495658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:29.496332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.498545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.499171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.503012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:29.503715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.503993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.504316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/e8708f9d98754ff88ae858e35d39eb29/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.510303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a44082f1e4291b18bf0735dba8291, user_id=e8708f9d98754ff88ae858e35d39eb29, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:29.511830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:29.521641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:29.526663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:29.528316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:29.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 121/229] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:29.541441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 109/230] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:29 2022] PUT /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/e8708f9d98754ff88ae858e35d39eb29/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 00:59:29.541974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:29.673409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45441e-90bc-4da2-a7b8-f03ec74ddeb4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:29.681756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:29.710472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:29.711047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.711395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.711954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.712707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:29.713111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:29.714555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:29.724256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 122/231] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:29.732103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:29.869927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e123-ac1d-43d6-8493-27a65ca1bff5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:29.906853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:29.907525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:29.907787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:29.908067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] PATH_INFO: `/v3/users/e8708f9d98754ff88ae858e35d39eb29` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:29.909276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] RBAC: Authorizing `identity:get_user(user_id=e8708f9d98754ff88ae858e35d39eb29)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:29.910726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:29.912453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 110/232] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:29 2022] GET /identity/v3/users/e8708f9d98754ff88ae858e35d39eb29 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:29.918482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:30.030587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34648-ab4d-42b6-b4ae-1db2ac18cc24 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:30.071617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c718b6-8f00-4d37-9d44-9c8bba5f419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:30.072368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c718b6-8f00-4d37-9d44-9c8bba5f419b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:30.072651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c718b6-8f00-4d37-9d44-9c8bba5f419b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:30.072931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c718b6-8f00-4d37-9d44-9c8bba5f419b None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:30.074122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c718b6-8f00-4d37-9d44-9c8bba5f419b None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:30.075618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c718b6-8f00-4d37-9d44-9c8bba5f419b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:30.077547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 123/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:29 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:30.083185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:30.113961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:30.246070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169640cb-29e0-4da8-a1dc-3f76031fb468 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:30.305621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fdeb8-eda3-4d68-8c2b-f1a22cde65b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:30.306866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fdeb8-eda3-4d68-8c2b-f1a22cde65b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:30.307360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fdeb8-eda3-4d68-8c2b-f1a22cde65b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:30.307859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fdeb8-eda3-4d68-8c2b-f1a22cde65b5 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/e8708f9d98754ff88ae858e35d39eb29/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:30.316945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fdeb8-eda3-4d68-8c2b-f1a22cde65b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a44082f1e4291b18bf0735dba8291, user_id=e8708f9d98754ff88ae858e35d39eb29, role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:30.318800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fdeb8-eda3-4d68-8c2b-f1a22cde65b5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:30.347500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 111/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:30 2022] PUT /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/e8708f9d98754ff88ae858e35d39eb29/roles/8fc2a014f1e24e08a63b265436f41eec => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:30.636022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99f32-ff23-4318-aed2-f6404435fa42 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:30.636220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99f32-ff23-4318-aed2-f6404435fa42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:30.636488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99f32-ff23-4318-aed2-f6404435fa42 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:30.638237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 124/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:30.648981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774677c-c895-4ddb-bc10-60809f8c2bbd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:30.649627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774677c-c895-4ddb-bc10-60809f8c2bbd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:30.650120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774677c-c895-4ddb-bc10-60809f8c2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:30.675425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3774677c-c895-4ddb-bc10-60809f8c2bbd None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:30.676654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3774677c-c895-4ddb-bc10-60809f8c2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:30.769504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 112/236] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:30.808399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99f32-ff23-4318-aed2-f6404435fa42 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:30.865156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:30.925231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcad6b8-d2e0-42ed-8a09-41b07a80a444 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:30.925768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcad6b8-d2e0-42ed-8a09-41b07a80a444 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:30.925768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcad6b8-d2e0-42ed-8a09-41b07a80a444 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:30.927681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 113/237] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:30.933078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b55109-b916-4579-896d-b3b659308181 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:30.933689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b55109-b916-4579-896d-b3b659308181 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:30.934037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b55109-b916-4579-896d-b3b659308181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:30.946808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99f32-ff23-4318-aed2-f6404435fa42 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:30.975170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d1b55109-b916-4579-896d-b3b659308181 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:30.978985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d1b55109-b916-4579-896d-b3b659308181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:30.986560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:31.018208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.019276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.019798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.020080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.046196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:31.053255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:31.054130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:31.054929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:31.060167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 125/238] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:31.065034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.111859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 114/239] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:31.156477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b55109-b916-4579-896d-b3b659308181 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.188335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:31.194682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e891131-ae6d-467f-9ef7-f0ef7d212c5c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.230819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.231801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.232234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.232648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.233519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:31.233969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:31.235543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:31.244884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 126/240] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:31.254968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.361916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b55109-b916-4579-896d-b3b659308181 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.424068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79506f-5dbe-4cf9-8186-5db35f39d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.424929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79506f-5dbe-4cf9-8186-5db35f39d722 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.425278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79506f-5dbe-4cf9-8186-5db35f39d722 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.425567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79506f-5dbe-4cf9-8186-5db35f39d722 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.426739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79506f-5dbe-4cf9-8186-5db35f39d722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:31.427510 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bfac7-f5ad-4e8f-a028-a780167103c1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.428243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79506f-5dbe-4cf9-8186-5db35f39d722 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:31.430105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 115/241] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:31.436296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897bd3-b3b2-49c9-bb5b-35205eaaed07 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.436710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897bd3-b3b2-49c9-bb5b-35205eaaed07 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.436858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897bd3-b3b2-49c9-bb5b-35205eaaed07 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.438722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 116/242] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:31.443634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897bd3-b3b2-49c9-bb5b-35205eaaed07 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.459200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:31.487891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdaaee-18e5-4326-8945-c05ba9cabf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.488892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdaaee-18e5-4326-8945-c05ba9cabf6c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.489190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdaaee-18e5-4326-8945-c05ba9cabf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.489475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdaaee-18e5-4326-8945-c05ba9cabf6c None admin] PATH_INFO: `/v3/users/f74ba4c880f843ae8f80d175ab863f66` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.509317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdaaee-18e5-4326-8945-c05ba9cabf6c None admin] RBAC: Authorizing `identity:get_user(user_id=f74ba4c880f843ae8f80d175ab863f66)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:31.510969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdaaee-18e5-4326-8945-c05ba9cabf6c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:31.514064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 127/243] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:31 2022] GET /identity/v3/users/f74ba4c880f843ae8f80d175ab863f66 => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:31.516392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.516943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.517398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.539338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:31.541000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:31.553773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:31.577043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897bd3-b3b2-49c9-bb5b-35205eaaed07 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.611671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.612360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.612644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.612917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.613888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:31.616248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:31.619599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 128/244] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:31.620816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.643926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 117/245] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:31 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:31.654137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.728140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:31.738968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c29d1-b839-4d91-b11f-525e68f6421e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.773190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b166e-9d8f-4020-af50-5a60e15f1973 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.779987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052295d-7913-4524-9e64-3da64153341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.780912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052295d-7913-4524-9e64-3da64153341f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.781342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052295d-7913-4524-9e64-3da64153341f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.781819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052295d-7913-4524-9e64-3da64153341f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.799950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:31.814328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:31.815128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:31.815407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:31.815683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:31.828571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:31.834564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:31.835614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:31.836424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 118/246] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:31.843832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:31.871548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052295d-7913-4524-9e64-3da64153341f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:31.873440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052295d-7913-4524-9e64-3da64153341f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:31.961314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63617c16-de53-46e1-9c0a-8f28896f45da None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:31.988647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:32.002049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.002894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.003550 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.003659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.004458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:32.005200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:32.006779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:32.017396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 119/247] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:32.023077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.032069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 129/248] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:32.039376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052295d-7913-4524-9e64-3da64153341f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.102639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:32.179627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcbad4-d2da-4b63-a53e-caff93bd1e21 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:32.186367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052295d-7913-4524-9e64-3da64153341f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:32.230586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99962708-ea43-4861-aee0-ea36421848a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.231301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99962708-ea43-4861-aee0-ea36421848a9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.232186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99962708-ea43-4861-aee0-ea36421848a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.232186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99962708-ea43-4861-aee0-ea36421848a9 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.234218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99962708-ea43-4861-aee0-ea36421848a9 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:32.234580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99962708-ea43-4861-aee0-ea36421848a9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:32.236630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 130/249] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:32 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:32.237070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242c306-1ea3-4a6f-be59-9fb46be18101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.237747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242c306-1ea3-4a6f-be59-9fb46be18101 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.237860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd1810-da58-4156-92bd-b66693e17dfe None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.238023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242c306-1ea3-4a6f-be59-9fb46be18101 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.238115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd1810-da58-4156-92bd-b66693e17dfe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.238296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242c306-1ea3-4a6f-be59-9fb46be18101 None admin] PATH_INFO: `/v3/users/71c019b9826c4539b97215ac8535a1ae` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.238389 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd1810-da58-4156-92bd-b66693e17dfe None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.239764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 131/250] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:32.244289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242c306-1ea3-4a6f-be59-9fb46be18101 None admin] RBAC: Authorizing `identity:get_user(user_id=71c019b9826c4539b97215ac8535a1ae)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:32.244499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd1810-da58-4156-92bd-b66693e17dfe None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.245757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242c306-1ea3-4a6f-be59-9fb46be18101 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:32.247709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 120/251] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:32 2022] GET /identity/v3/users/71c019b9826c4539b97215ac8535a1ae => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:32.255259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.255580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.255863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.275360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:32.276592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:32.341390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:32.361495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 121/252] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:32.362372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.373904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd1810-da58-4156-92bd-b66693e17dfe None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:32.391262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:32.422412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.423091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.423383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.423661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.424481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] RBAC: Adding query filter params (role.id=639f5d438fa44b96b0e715bcd0eab26a, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=f74ba4c880f843ae8f80d175ab863f66) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:32.424795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:32.426287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:32.432644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 132/253] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:32 2022] GET /identity/v3/role_assignments?user.id=f74ba4c880f843ae8f80d175ab863f66&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=639f5d438fa44b96b0e715bcd0eab26a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:32.433496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.554737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641b7e7-49f9-4cfc-8cce-a9a83f025acd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:32.595967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.596657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.596948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.597233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.598526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:32.600120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:32.602112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 122/254] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:32 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:32.612444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.680039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:32.682957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26776ac3-d23f-42a3-8245-2bee30a371cb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:32.742955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95705781-f65f-4c36-b467-dfc1d75f65da None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:32.755850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.756452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.756745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.757028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.772581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:32.778221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:32.779146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:32.779937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:32.785829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 133/255] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:32.787927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:32.788940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:32.789452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:32.789930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/71c019b9826c4539b97215ac8535a1ae/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:32.792086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:32.792939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a44082f1e4291b18bf0735dba8291, user_id=71c019b9826c4539b97215ac8535a1ae, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:32.794702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:32.812625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 123/256] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:32 2022] PUT /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/71c019b9826c4539b97215ac8535a1ae/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:32.838584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:32.952951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b02a59-a668-42eb-a98a-b669000f7add None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:33.002095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:33.002762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:33.003077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:33.003357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:33.004119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:33.004430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:33.006244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460b8c4-d8b7-4779-a664-fdf95ec824ba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:33.016604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 134/257] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:33.022269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:33.168558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef2f33-c13c-47ed-8d18-5efa6fee725d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:33.169084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef2f33-c13c-47ed-8d18-5efa6fee725d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:33.169159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef2f33-c13c-47ed-8d18-5efa6fee725d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:33.170735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 135/258] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:33.176177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:33.176457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:33.176735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:33.211078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:33.212343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:33.236136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894bc2f-a59c-43aa-8dd0-f3cf8c0db21f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:33.265370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:33.323309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:33.324304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:33.324602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:33.324864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] PATH_INFO: `/v3/users/e8708f9d98754ff88ae858e35d39eb29` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:33.327171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] RBAC: Authorizing `identity:get_user(user_id=e8708f9d98754ff88ae858e35d39eb29)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:33.329657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:33.334657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 124/259] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:33 2022] GET /identity/v3/users/e8708f9d98754ff88ae858e35d39eb29 => generated 283 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:33.341466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:33.380661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 136/260] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:33.408552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:33.462055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:33.557049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e1e37-382f-47a0-a4d4-b6fc8c160659 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:33.607773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140702f-b334-41f2-9b73-26fad00bd9b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:33.625349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:33.626008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:33.626294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:33.626573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:33.679624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:33.680307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:33.680600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:33.680877 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:33.686634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:33.699107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:33.704131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:33.704973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:33.705826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:33.711112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 137/261] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:33.711374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:33.712175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:33.713790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:33.839606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 125/262] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:33.847326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:33.878014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:33.913004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfd5c-bc05-40c0-8cbc-4cf27cffab59 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:33.944165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:33.968412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6641e54-e952-447a-a883-8ed84c09ba26 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:34.000656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:34.001246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:34.001626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:34.002018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:34.002856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:34.003287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:34.005137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:34.014187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:34.015256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:34.015714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:34.016811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:34.019593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:34.021690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 138/263] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:34.023423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:34.025742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 126/264] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:33 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:34.031659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:34.031890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:34.201024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6969c1a-f73d-44a8-919b-64719ffdc454 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:34.209393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e61c-9532-427f-b85e-239f1da6418e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:34.250523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:34.251582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:34.252095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:34.252644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] PATH_INFO: `/v3/users/e8708f9d98754ff88ae858e35d39eb29` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:34.256958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] RBAC: Authorizing `identity:get_user(user_id=e8708f9d98754ff88ae858e35d39eb29)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:34.258853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:34.261185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 127/265] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:34 2022] GET /identity/v3/users/e8708f9d98754ff88ae858e35d39eb29 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:34.263682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:34.264416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:34.264768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:34.265112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:34.266103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=e8708f9d98754ff88ae858e35d39eb29) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:34.266513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:34.267531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:34.268316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:34.275760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 139/266] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:34 2022] GET /identity/v3/role_assignments?user.id=e8708f9d98754ff88ae858e35d39eb29&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:34.301692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:34.401446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087abc-f96e-48a0-98b1-730fecc51203 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:34.449186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:34.450140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:34.450555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:34.450987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:34.546827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:34.548277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:34.568311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:34.664934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 128/267] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:34.670319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:34.701521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:34.824861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821aead-8685-4e27-8d14-604d87b97222 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:34.873714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c29b-b799-4d67-b697-8a731215ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:34.874387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c29b-b799-4d67-b697-8a731215ba4a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:34.874676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c29b-b799-4d67-b697-8a731215ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:34.874981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c29b-b799-4d67-b697-8a731215ba4a None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:34.880003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185c29b-b799-4d67-b697-8a731215ba4a None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:34.881508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185c29b-b799-4d67-b697-8a731215ba4a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:34.883621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 140/268] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:34 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:34.890466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.011898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49fd94-45da-42ed-9066-1434c0618a11 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.012223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49fd94-45da-42ed-9066-1434c0618a11 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.012465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49fd94-45da-42ed-9066-1434c0618a11 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.014081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 141/269] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:35.020829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.021128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.021404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.053408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:35.055059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:35.070194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baafd3e-726c-4fd5-8d5e-820c9ecf1343 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:35.099313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:35.108628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:35.109369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.109664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.109947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.110888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] RBAC: Adding query filter params (role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=e8708f9d98754ff88ae858e35d39eb29) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:35.111220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:35.112716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975bb12-81cb-4440-8950-0a8711db2c52 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:35.119268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 129/270] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:34 2022] GET /identity/v3/role_assignments?user.id=e8708f9d98754ff88ae858e35d39eb29&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:35.180621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e72f-436a-4112-810a-cc0fa4b871db None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.180912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e72f-436a-4112-810a-cc0fa4b871db None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.181196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e72f-436a-4112-810a-cc0fa4b871db None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.183047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 130/271] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:35.187113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 142/272] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:35.190242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.190742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.191220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.212129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:35.213727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:35.244134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.263253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:35.312708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 131/273] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:35.336259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.407897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca0bb2-ab0d-4584-8838-54217e63f729 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:35.450453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf478-6e99-4d8c-8eff-d6969f1c5172 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:35.464754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:35.494506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:35.495694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.496208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.496769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.498200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:35.499915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:35.502057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 143/274] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:35.507498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.510009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:35.511030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.511766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.512264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.527528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:35.534253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:35.535174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:35.535958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:35.538405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 132/275] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:35.544665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.616372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:35.640989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39feadb3-7f59-4152-8d03-566c4ef42c96 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:35.683278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:35.683464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a91afb-39b3-454e-81d2-562057ac9074 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:35.684433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.684939 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.685422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.686323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:35.688193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:35.716764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 144/276] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 00:59:35 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:35.731549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:35.732762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.733280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.733800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.734819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:35.735397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:35.737208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ad3ed-a4ff-44b2-a8db-98f49320f122 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:35.748407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 133/277] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:35.755239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.789660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021770-0b6f-4440-9f0f-68e4907bc72c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.790298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021770-0b6f-4440-9f0f-68e4907bc72c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.790865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021770-0b6f-4440-9f0f-68e4907bc72c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.793542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 134/278] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:35.798881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df6c8d-c8fd-430d-bd2c-ff6bb4a5ee42 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.799224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df6c8d-c8fd-430d-bd2c-ff6bb4a5ee42 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.799451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df6c8d-c8fd-430d-bd2c-ff6bb4a5ee42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.826376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b1df6c8d-c8fd-430d-bd2c-ff6bb4a5ee42 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:35.828610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b1df6c8d-c8fd-430d-bd2c-ff6bb4a5ee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:35.866156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:35.895407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ba34-6c29-4f1c-b879-e178887b66f9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:35.926103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 135/279] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:35.936062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ba099-8d34-4cac-9e18-27defddd98a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:35.936823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ba099-8d34-4cac-9e18-27defddd98a4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.937126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ba099-8d34-4cac-9e18-27defddd98a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.937412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ba099-8d34-4cac-9e18-27defddd98a4 None admin] PATH_INFO: `/v3/users/859e2995a548494d9d5d6f7a7c2138bc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.947229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:35.952384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a10fd3-19a3-442d-960b-3ad8f44f8ada None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.952682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a10fd3-19a3-442d-960b-3ad8f44f8ada None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.952966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a10fd3-19a3-442d-960b-3ad8f44f8ada None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.954342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 136/280] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:35.955169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ba099-8d34-4cac-9e18-27defddd98a4 None admin] RBAC: Authorizing `identity:get_user(user_id=859e2995a548494d9d5d6f7a7c2138bc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:35.956277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a10fd3-19a3-442d-960b-3ad8f44f8ada None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:35.957003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ba099-8d34-4cac-9e18-27defddd98a4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:35.959013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 145/281] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:35 2022] GET /identity/v3/users/859e2995a548494d9d5d6f7a7c2138bc => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:35.962411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:35.963034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:35.963491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:35.987001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:35.988477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:36.072103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 146/282] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:36.072824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.085497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a10fd3-19a3-442d-960b-3ad8f44f8ada None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.133958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.134713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.135053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.135337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.151855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:36.159453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:36.160572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:36.161429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 137/283] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:36.162071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.195694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:36.203975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ed705-4252-4a97-bacb-5448a51f2af3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.236960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:36.248572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.249217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.250199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.250483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.251362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, user.id=859e2995a548494d9d5d6f7a7c2138bc, scope.system=all) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:36.251678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:36.253364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:36.259938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 147/284] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/role_assignments?user.id=859e2995a548494d9d5d6f7a7c2138bc&scope.system=all&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:36.271501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.306130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fe781-a89e-4577-811e-c5ae8ace14e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.372085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.373171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.380129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.380751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.395135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:36.401299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:36.402245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:36.403073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:36.406114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 138/285] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:36.413785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.522061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:36.535543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040be9fc-1e67-476e-8145-bb1ff0fafc19 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.542620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5630e2-43ae-46d8-9fc2-144bfa3727c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.589757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.590577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.598857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.598857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.598857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:36.598857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:36.598857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:36.603822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 139/286] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:36.610569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.611518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.611626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.612253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.612772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.613797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:36.614466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:36.616373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:36.626712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 148/287] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:36.640078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.755467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:36.782973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e90e9-905c-4fbd-aa10-8d4e8836e6f0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.818548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.819251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.819542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.819826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] PATH_INFO: `/v3/users/71c019b9826c4539b97215ac8535a1ae` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.821049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] RBAC: Authorizing `identity:get_user(user_id=71c019b9826c4539b97215ac8535a1ae)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:36.822977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:36.825114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 140/288] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/users/71c019b9826c4539b97215ac8535a1ae => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:36.831353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.855323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:36.908802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af237a-cc21-4b11-9fae-bbf0bfced33b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:36.953078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:36.953739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:36.954054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:36.954333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] PATH_INFO: `/v3/users/f74ba4c880f843ae8f80d175ab863f66` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:36.955868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] RBAC: Authorizing `identity:get_user(user_id=f74ba4c880f843ae8f80d175ab863f66)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:36.957409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:36.959232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 149/289] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/users/f74ba4c880f843ae8f80d175ab863f66 => generated 297 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:36.964852 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:36.971502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc06301-8035-4b19-9edf-26e9bdb6e46c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:37.044313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:37.044871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:37.045162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:37.045445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:37.076257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120048f-29f2-4142-8ccb-c8400d42b732 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:37.116015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:37.149719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:37.150459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:37.150737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:37.151029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:37.152402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:37.153868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:37.155688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 150/290] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:37.166081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:37.222591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:37.224155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:37.256663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:37.397307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 141/291] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:37.403113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:37.437164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622cbbb-dc4f-4f3e-adea-6830f5bd6c43 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:37.488434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:37.489147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:37.489436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:37.489719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/f74ba4c880f843ae8f80d175ab863f66/roles/639f5d438fa44b96b0e715bcd0eab26a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:37.491761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a44082f1e4291b18bf0735dba8291, user_id=f74ba4c880f843ae8f80d175ab863f66, role_id=639f5d438fa44b96b0e715bcd0eab26a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:37.493225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:37.509276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 151/292] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 00:59:37 2022] PUT /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291/users/f74ba4c880f843ae8f80d175ab863f66/roles/639f5d438fa44b96b0e715bcd0eab26a => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:37.569989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1523be8-e662-4595-b443-9c4739450c10 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:37.584346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:37.614562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37049ccc-af18-4a9d-85ed-c94cf8dbe23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:37.615276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37049ccc-af18-4a9d-85ed-c94cf8dbe23d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:37.615560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37049ccc-af18-4a9d-85ed-c94cf8dbe23d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:37.615837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37049ccc-af18-4a9d-85ed-c94cf8dbe23d None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:37.619803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37049ccc-af18-4a9d-85ed-c94cf8dbe23d None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:37.621307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37049ccc-af18-4a9d-85ed-c94cf8dbe23d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:37.623465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 142/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:37 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:37.630187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:37.705294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:37.772451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b9853-edce-4e0f-a8de-de8e0adbe523 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:37.817689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:37.818340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:37.818628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:37.818916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:37.819728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=71c019b9826c4539b97215ac8535a1ae) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:37.820037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:37.821503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd0c07-f030-4eb1-a968-666595729d80 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:37.828467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 152/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:37 2022] GET /identity/v3/role_assignments?user.id=71c019b9826c4539b97215ac8535a1ae&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:38.181155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dd896-ce39-46a2-be50-7a2acf9cac1f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.181639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dd896-ce39-46a2-be50-7a2acf9cac1f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.181809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dd896-ce39-46a2-be50-7a2acf9cac1f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.183219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 143/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:38.188745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61e235-1516-47ac-908c-8201bed29e5f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.189036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61e235-1516-47ac-908c-8201bed29e5f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.189306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61e235-1516-47ac-908c-8201bed29e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.211466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7b61e235-1516-47ac-908c-8201bed29e5f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:38.212834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b61e235-1516-47ac-908c-8201bed29e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:38.294410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 153/296] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:38.318966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dd896-ce39-46a2-be50-7a2acf9cac1f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:38.452804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dd896-ce39-46a2-be50-7a2acf9cac1f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:38.470677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a2d6-c053-450e-b330-d4035f0cce04 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.471231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a2d6-c053-450e-b330-d4035f0cce04 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.471304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a2d6-c053-450e-b330-d4035f0cce04 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.474714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 154/297] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:38.480616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.481146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.481591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.501912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:38.502493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:38.503228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.503514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.503838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.504457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:38.518265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:38.524862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:38.526099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:38.526905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 144/298] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:38.532205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:38.555702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:38.588914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:38.606940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 155/299] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:38.628888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:38.665088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b8aa6-111c-4ee9-9cd3-40374cdd7a89 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:38.705321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:38.705879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.706164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.706441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.707220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:38.707757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:38.709275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adddfa9-4765-4f57-8a6b-70f86e38b9fa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:38.718354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 145/300] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:38.719745 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eed118-b3ad-4e58-a7c9-ae0bb49ee126 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.720792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eed118-b3ad-4e58-a7c9-ae0bb49ee126 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.720792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eed118-b3ad-4e58-a7c9-ae0bb49ee126 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.726881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 146/301] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:38.726881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eed118-b3ad-4e58-a7c9-ae0bb49ee126 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:38.770199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d98df0-4d9c-4f94-ad5a-2facbad545fa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:38.807823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41fcc-71d8-4834-92e9-6573d0c83900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:38.808498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41fcc-71d8-4834-92e9-6573d0c83900 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.808790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41fcc-71d8-4834-92e9-6573d0c83900 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.809119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41fcc-71d8-4834-92e9-6573d0c83900 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.810280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a41fcc-71d8-4834-92e9-6573d0c83900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:38.812328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a41fcc-71d8-4834-92e9-6573d0c83900 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:38.814989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 156/302] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:38.816270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.816565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.816845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.836792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:38.838068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:38.850125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eed118-b3ad-4e58-a7c9-ae0bb49ee126 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:38.852475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:38.879585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:38.909081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:38.910138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:38.910602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:38.911069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] PATH_INFO: `/v3/users/fa3c17afb6144aaaa5b6192a70bc6579` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:38.927479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 157/303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:38.928794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:38.936053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] RBAC: Authorizing `identity:get_user(user_id=fa3c17afb6144aaaa5b6192a70bc6579)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:38.937859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:38.944935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 147/304] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:38 2022] GET /identity/v3/users/fa3c17afb6144aaaa5b6192a70bc6579 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:38.946203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:39.052286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953229e-cd77-46e0-a1a0-1bb6e1c3d3fe None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:39.068653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22189b-d652-47b3-ac6a-e56d7ba835f3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:39.089052 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:39.089697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.089983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.090252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] PATH_INFO: `/v3/groups` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.090961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:39.092419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:39.108013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:39.109000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.109463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.109925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.110932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] RBAC: Adding query filter params (role.id=639f5d438fa44b96b0e715bcd0eab26a, user.id=fa3c17afb6144aaaa5b6192a70bc6579, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:39.111118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 158/305] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 00:59:38 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 00:59:39.111589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:39.111947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:39.113512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:39.122932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 148/306] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 00:59:38 2022] GET /identity/v3/role_assignments?user.id=fa3c17afb6144aaaa5b6192a70bc6579&scope.system=all&role.id=639f5d438fa44b96b0e715bcd0eab26a => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:39.176410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:39.267841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c48e-08fb-4ee9-9630-67148a36d5b9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:39.316017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:39.316671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.316960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.317235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.331152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:39.336192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:39.337135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:39.338687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 159/307] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:39.344030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:39.373561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:39.462130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e258e-2ac9-43f3-a96b-8aa780677125 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:39.500246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc680-9010-4734-83aa-716995e01271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:39.500886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc680-9010-4734-83aa-716995e01271 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.501172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc680-9010-4734-83aa-716995e01271 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.501447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc680-9010-4734-83aa-716995e01271 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.502186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc680-9010-4734-83aa-716995e01271 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:39.502496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc680-9010-4734-83aa-716995e01271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:39.503929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc680-9010-4734-83aa-716995e01271 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:39.514326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 149/308] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:39.520286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:39.634983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:39.646359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83186031-e3c9-4f8b-8558-16fd812cfb5b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:39.686491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09830cc6-7a59-489b-a103-8b52ff867003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:39.687649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09830cc6-7a59-489b-a103-8b52ff867003 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.688145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09830cc6-7a59-489b-a103-8b52ff867003 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.688630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09830cc6-7a59-489b-a103-8b52ff867003 None admin] PATH_INFO: `/v3/users/859e2995a548494d9d5d6f7a7c2138bc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.690818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09830cc6-7a59-489b-a103-8b52ff867003 None admin] RBAC: Authorizing `identity:get_user(user_id=859e2995a548494d9d5d6f7a7c2138bc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:39.692720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09830cc6-7a59-489b-a103-8b52ff867003 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:39.694866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 160/309] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:39 2022] GET /identity/v3/users/859e2995a548494d9d5d6f7a7c2138bc => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:39.700559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc680-9010-4734-83aa-716995e01271 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:39.798043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:39.817444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc680-9010-4734-83aa-716995e01271 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:39.823856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90d460-251b-4a69-a162-5f20bc2f157e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.824464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90d460-251b-4a69-a162-5f20bc2f157e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.824937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90d460-251b-4a69-a162-5f20bc2f157e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.826532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 161/310] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:39.832089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.832646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.833119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.858070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:39.859431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:39.862499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:39.863327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:39.863682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:39.864054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] PATH_INFO: `/v3/system/users/859e2995a548494d9d5d6f7a7c2138bc/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:39.865752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=859e2995a548494d9d5d6f7a7c2138bc, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:39.867307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:39.879416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 150/311] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 00:59:39 2022] PUT /identity/v3/system/users/859e2995a548494d9d5d6f7a7c2138bc/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:39.949607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 162/312] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:39.972030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:40.093425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aaba6c-090f-43d0-8852-10df28a528d8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:40.128879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:40.129546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:40.130196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:40.130466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:40.141740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:40.148658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:40.149604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:40.150476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 151/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:40.159343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:40.260207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:40.273805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34da23-4dfa-4fb4-87b1-c650b334a152 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:40.310681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:40.311360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:40.311645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:40.311928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:40.312672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:40.312985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:40.314402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:40.323776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 163/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:40.331863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:40.347690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:40.464434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa233-e532-408d-8a50-56003ee5f110 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:40.507358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:40.508422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:40.508912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:40.509388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] PATH_INFO: `/v3/users/f74ba4c880f843ae8f80d175ab863f66` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:40.510971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] RBAC: Authorizing `identity:get_user(user_id=f74ba4c880f843ae8f80d175ab863f66)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:40.512649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:40.515007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 152/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:40 2022] GET /identity/v3/users/f74ba4c880f843ae8f80d175ab863f66 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:40.523395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:40.624647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375609c8-8810-4422-beff-0b86bc0d60cd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:40.661462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:40.662108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:40.662398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:40.662677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:40.695308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:40.740332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:40.741827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:40.844805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 164/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:40.850414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:40.939507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:40.964683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e176990-a013-4492-9530-c9214003d0e5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:41.003733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb278c5b-d9d6-4f09-ab87-9346a2bc2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:41.005150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb278c5b-d9d6-4f09-ab87-9346a2bc2bfe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.005647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb278c5b-d9d6-4f09-ab87-9346a2bc2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.006125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb278c5b-d9d6-4f09-ab87-9346a2bc2bfe None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.007586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb278c5b-d9d6-4f09-ab87-9346a2bc2bfe None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:41.009255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb278c5b-d9d6-4f09-ab87-9346a2bc2bfe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:41.011220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 153/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:40 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:41.019115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:41.070261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:41.123118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25357c-219f-490a-9a53-a0c77f3cda1e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:41.160346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:41.160992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.161278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.161551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.162367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] RBAC: Adding query filter params (role.id=639f5d438fa44b96b0e715bcd0eab26a, scope.project.id=5a6a44082f1e4291b18bf0735dba8291, user.id=f74ba4c880f843ae8f80d175ab863f66) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:41.162677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:41.164207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb188b3d-46b2-47c8-8191-a919b3a9db39 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:41.172422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 165/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 00:59:41 2022] GET /identity/v3/role_assignments?user.id=f74ba4c880f843ae8f80d175ab863f66&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=639f5d438fa44b96b0e715bcd0eab26a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:41.293557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6053-cd93-45d1-a861-656903b168aa None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.293911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6053-cd93-45d1-a861-656903b168aa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.294127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6053-cd93-45d1-a861-656903b168aa None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.295759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 154/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:41.301460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.301807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.302008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.322174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:41.323435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:41.344916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2880c-3117-434e-b4e0-b111c033987d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.345207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2880c-3117-434e-b4e0-b111c033987d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.345483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2880c-3117-434e-b4e0-b111c033987d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.347004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 155/320] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:41.352084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.352676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.352676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.374283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:41.375435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:41.417806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 166/321] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:41.446983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:41.467123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 156/322] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:41.494392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:41.518507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:41.585323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:41.589488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7975a-059d-43ad-bae9-4ed2ef3b7235 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:41.612694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c63c36-b93d-4126-a2d0-34c041b46871 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:41.642780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:41.643403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.643691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.643973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.645171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:41.646634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:41.648306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 167/323] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 00:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:41.654106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:41.661545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:41.662219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.662505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.662806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.675424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:41.681409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:41.682347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:41.683189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 157/324] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:41.688770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:41.768363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d886191-d879-4c2c-9945-06c16aad78fb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:41.783072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:41.796916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ce454-c6bd-4a69-a840-6849953aac8e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:41.806024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:41.806918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.807203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.807488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] PATH_INFO: `/v3/groups` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.808172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:41.809617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:41.830758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 168/325] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jul 28 00:59:41 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 00:59:41.846603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:41.847309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:41.847595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:41.847879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:41.848679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:41.848973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:41.850447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:41.853771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:41.862920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 158/326] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:41.869170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:42.024883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acd5a-2ddc-4264-8623-01de26304caf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:42.062178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db195bd6-9f86-4cbc-91bc-5c46daa7bc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:42.062948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195bd6-9f86-4cbc-91bc-5c46daa7bc08 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.063197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195bd6-9f86-4cbc-91bc-5c46daa7bc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.063420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195bd6-9f86-4cbc-91bc-5c46daa7bc08 None admin] PATH_INFO: `/v3/users/fa3c17afb6144aaaa5b6192a70bc6579` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.064667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db195bd6-9f86-4cbc-91bc-5c46daa7bc08 None admin] RBAC: Authorizing `identity:get_user(user_id=fa3c17afb6144aaaa5b6192a70bc6579)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:42.066142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db195bd6-9f86-4cbc-91bc-5c46daa7bc08 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:42.067858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 169/327] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:41 2022] GET /identity/v3/users/fa3c17afb6144aaaa5b6192a70bc6579 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:42.073629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:42.088176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42cb97-f87b-4e61-a169-e5059dc1dac9 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.088471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42cb97-f87b-4e61-a169-e5059dc1dac9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.088750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42cb97-f87b-4e61-a169-e5059dc1dac9 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.090413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 170/328] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:42.095951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.096238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.096512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.120759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:42.121969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:42.170043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:42.222474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 171/329] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:42.223033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de70a50-fa50-4dc7-8eda-acc91a080b66 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:42.251599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:42.268057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:42.269145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.269632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.270137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] PATH_INFO: `/v3/system/users/fa3c17afb6144aaaa5b6192a70bc6579/roles/639f5d438fa44b96b0e715bcd0eab26a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.271917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fa3c17afb6144aaaa5b6192a70bc6579, role_id=639f5d438fa44b96b0e715bcd0eab26a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:42.273554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:42.287144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 159/330] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jul 28 00:59:42 2022] PUT /identity/v3/system/users/fa3c17afb6144aaaa5b6192a70bc6579/roles/639f5d438fa44b96b0e715bcd0eab26a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:42.386927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd733a-4c57-41ca-8014-120709fa5baa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:42.433719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:42.434769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.435266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.435750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.453823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:42.459479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:42.460415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:42.461258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:42.463998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 172/331] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:42.469989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:42.553472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:42.603160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10264d-975d-4726-b413-20c5d0dd11a2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:42.645901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:42.646551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.646845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.647139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.647885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:42.648190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:42.649585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:42.659083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 160/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:42.664817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:42.701078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:42.782157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc628af-f94e-49ca-a616-74b7d6a33203 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:42.825625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ac9db-896c-4e15-9837-0af8a7187400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:42.826233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ac9db-896c-4e15-9837-0af8a7187400 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:42.826515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ac9db-896c-4e15-9837-0af8a7187400 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:42.826790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ac9db-896c-4e15-9837-0af8a7187400 None admin] PATH_INFO: `/v3/users/859e2995a548494d9d5d6f7a7c2138bc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:42.828189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ac9db-896c-4e15-9837-0af8a7187400 None admin] RBAC: Authorizing `identity:get_user(user_id=859e2995a548494d9d5d6f7a7c2138bc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:42.829650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ac9db-896c-4e15-9837-0af8a7187400 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:42.831509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 173/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:42 2022] GET /identity/v3/users/859e2995a548494d9d5d6f7a7c2138bc => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:42.837181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:42.961754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f3096-009a-48ee-9d35-374ec7a36df5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:42.993617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:43.008906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:43.009783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:43.010330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:43.010804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:43.012358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] RBAC: Adding query filter params (role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, user.id=859e2995a548494d9d5d6f7a7c2138bc, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:43.012930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:43.015713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b567e44-cd32-44ca-8a28-08cf5d7290ff None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:43.036594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 161/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 00:59:42 2022] GET /identity/v3/role_assignments?user.id=859e2995a548494d9d5d6f7a7c2138bc&scope.system=all&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:44.149191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7a3c7-ed85-4fe3-90a5-278bbbf9cd39 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.149737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7a3c7-ed85-4fe3-90a5-278bbbf9cd39 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.150080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7a3c7-ed85-4fe3-90a5-278bbbf9cd39 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.151827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 174/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:44.157992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.158487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.158565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.183945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:44.189587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:44.267689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 162/336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:44.292794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7a3c7-ed85-4fe3-90a5-278bbbf9cd39 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:44.411125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:44.418539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7a3c7-ed85-4fe3-90a5-278bbbf9cd39 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:44.464714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:44.465390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.465672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.465946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.478403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:44.483501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:44.484398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:44.485183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 175/337] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:44.491506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:44.642099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85efe4-97b5-4204-bed7-a479531a44d6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:44.657771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:44.683573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:44.684125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.684407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.684684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.685416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:44.685724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:44.687118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ab11b-66b2-461b-8925-2625f8aff423 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:44.699154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 163/338] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:44.704637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:44.856964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ce8b1-16ac-463e-84ab-cc15a5065426 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:44.887792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301dc5-fa72-4575-bc30-944b8fa4b9e9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.888155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301dc5-fa72-4575-bc30-944b8fa4b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.888469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301dc5-fa72-4575-bc30-944b8fa4b9e9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.890029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 164/339] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:44.895341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:44.895988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.896219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ad805-650f-44c5-9189-748b1777f60f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:44.896333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.896505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ad805-650f-44c5-9189-748b1777f60f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:44.896613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.896769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ad805-650f-44c5-9189-748b1777f60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:44.933401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:44.952289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1c2ad805-650f-44c5-9189-748b1777f60f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:44.953435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1c2ad805-650f-44c5-9189-748b1777f60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:44.986012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:44.987549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:45.113128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 176/340] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:45.115570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 165/341] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:45.119591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.140453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fc07e-3278-406a-8619-e5b2b1b01d23 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.140788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fc07e-3278-406a-8619-e5b2b1b01d23 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.141100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fc07e-3278-406a-8619-e5b2b1b01d23 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.144227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 166/342] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:45.155785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.156079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.156363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.187985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:45.189216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:45.215818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:45.270967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57c029-ff80-4213-8842-28bdecd0102d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:45.285861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 167/343] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:45.286782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.314046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:45.315462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.315799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.316262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.317872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:45.320532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:45.322826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 177/344] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:45.328621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.364618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:45.471136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6cf8b-1d76-47a1-8232-f4321a424739 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:45.505891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:45.518826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:45.519633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.519991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.520270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.533704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:45.540130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:45.542292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 178/345] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:45.544327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.587162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb84aba-f1e9-46ff-b1b0-ac8656889188 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:45.680622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:45.681842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.682376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.682870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.711105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:45.716729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:45.717682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:45.718491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:45.721762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 168/346] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:45.725023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.759757 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66772-4140-48b5-8c1f-2ae7b2f3993b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:45.814374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:45.815643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.816153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.816685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.823964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:45.825849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:45.828287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 179/347] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:45.829298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.874569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:45.885359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6eaea-8034-4c4a-b93a-23152f966d35 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:45.900111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:45.932818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:45.933463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.933751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.934032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.934785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:45.935124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:45.936642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d45db2-d604-47a6-84af-14f27509fb66 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:45.947957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 169/348] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:45.949316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7cfb1-97c7-4c4a-be46-6f88a1c57cf7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:45.949605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7cfb1-97c7-4c4a-be46-6f88a1c57cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:45.949881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7cfb1-97c7-4c4a-be46-6f88a1c57cf7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:45.951345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 170/349] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:45.951944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7cfb1-97c7-4c4a-be46-6f88a1c57cf7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:45.966514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae7013-c328-4f58-8608-f9b2896312ce None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.006899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.007958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.008419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.008973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.009947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:46.010435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:46.012173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:46.022132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 180/350] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:46.023486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.071689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7cfb1-97c7-4c4a-be46-6f88a1c57cf7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.110343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.111205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.111381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.111660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.112469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5f2e064ab9374fbab081907969175e05) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:46.112778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:46.114393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4d359-a085-416d-8a02-1a4f57cec900 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:46.121234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 171/351] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:45 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:46.122590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822f728-1088-4382-8834-5772e21b149f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.122905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822f728-1088-4382-8834-5772e21b149f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.123186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822f728-1088-4382-8834-5772e21b149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.139161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54e698-5fb7-4541-914f-718a829200ef None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.144902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9822f728-1088-4382-8834-5772e21b149f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:46.146124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9822f728-1088-4382-8834-5772e21b149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:46.155112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:46.168252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:46.183774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.184912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.185375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.185829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] PATH_INFO: `/v3/users/fa3c17afb6144aaaa5b6192a70bc6579` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.187380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] RBAC: Authorizing `identity:get_user(user_id=fa3c17afb6144aaaa5b6192a70bc6579)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:46.189085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:46.191404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 181/352] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/users/fa3c17afb6144aaaa5b6192a70bc6579 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:46.192580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.272526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 172/353] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:46.273339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822f728-1088-4382-8834-5772e21b149f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.304669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87047e-e17e-4a5d-9304-e5a19f53713e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.343293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.344245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.344727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.345179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.426006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822f728-1088-4382-8834-5772e21b149f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.474568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.475347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.475821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.476234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.477210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] RBAC: Adding query filter params (role.id=639f5d438fa44b96b0e715bcd0eab26a, user.id=fa3c17afb6144aaaa5b6192a70bc6579, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:46.477672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:46.479532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:46.492415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:46.497122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 173/354] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/role_assignments?user.id=fa3c17afb6144aaaa5b6192a70bc6579&scope.system=all&role.id=639f5d438fa44b96b0e715bcd0eab26a => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:46.499143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.518572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:46.529174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:46.531082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:46.667910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 182/355] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:46.675976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.688828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0902be-9f19-4f17-a059-7309477bfb93 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.780167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.781599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.782279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.782992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.802514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:46.808164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:46.809795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:46.811367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 174/356] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:46.818719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.862993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:46.902725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d9b90-5483-4610-b2f7-adede151f267 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:46.927060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:46.957258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bb67d-13f1-43ed-9a75-411c546ddcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:46.958170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bb67d-13f1-43ed-9a75-411c546ddcad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.958625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bb67d-13f1-43ed-9a75-411c546ddcad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.959085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bb67d-13f1-43ed-9a75-411c546ddcad None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.960437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bb67d-13f1-43ed-9a75-411c546ddcad None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:46.961988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bb67d-13f1-43ed-9a75-411c546ddcad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:46.963970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 183/357] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:46.965426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1f9c2-6a8e-4123-9f78-223b80150b32 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:46.965859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1f9c2-6a8e-4123-9f78-223b80150b32 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:46.966284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1f9c2-6a8e-4123-9f78-223b80150b32 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:46.972284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 184/358] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:46.973730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1f9c2-6a8e-4123-9f78-223b80150b32 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:46.979544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628f65f-c190-460f-86f0-f058d92b2283 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.028327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.029295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.029817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.030335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.033079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:47.033614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:47.035258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0aba7-eaa1-47bf-a187-849fd8ed3c50 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:47.045042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 175/359] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:47.046675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.052479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.053162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.075503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:47.076825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:47.122797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1f9c2-6a8e-4123-9f78-223b80150b32 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.166786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 176/360] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:47.168193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.168373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.169048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.169316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.169597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.170789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:47.172534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:47.180090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 185/361] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:46 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:47.182472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.206734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:47.247551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:47.294711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f3de6-04a4-4cf6-bc5e-2c94c79bc43c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.313244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02183-7b66-4b19-9ca5-df7c8d995ff2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.340205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbbfa9-dc79-4176-ac58-fb397b31ce92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.341104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbbfa9-dc79-4176-ac58-fb397b31ce92 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.341558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbbfa9-dc79-4176-ac58-fb397b31ce92 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.341975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbbfa9-dc79-4176-ac58-fb397b31ce92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.356592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.357538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.357904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.358275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.359375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5f2e064ab9374fbab081907969175e05) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:47.359796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:47.362103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:47.375324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 186/362] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:47.376489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.431029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cbbfa9-dc79-4176-ac58-fb397b31ce92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:47.432894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cbbfa9-dc79-4176-ac58-fb397b31ce92 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:47.507101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:47.511470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:47.536415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cd70-4e27-48d9-a621-1a709e4a9be3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.559748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 177/363] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:47.560985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7ebe-7412-49c4-ac45-2614fa696d12 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.561277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7ebe-7412-49c4-ac45-2614fa696d12 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.561560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7ebe-7412-49c4-ac45-2614fa696d12 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.567229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 178/364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:47.568769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7ebe-7412-49c4-ac45-2614fa696d12 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.579334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8260dd-fa30-4771-aceb-aa990dbbc299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.580169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8260dd-fa30-4771-aceb-aa990dbbc299 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.580566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8260dd-fa30-4771-aceb-aa990dbbc299 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.580933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8260dd-fa30-4771-aceb-aa990dbbc299 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.595285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8260dd-fa30-4771-aceb-aa990dbbc299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-eb8260dd-fa30-4771-aceb-aa990dbbc299 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:47.600615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:47.602401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:47.604683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:47.607040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 187/365] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:47.607040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.607040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.607040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.631186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:47.632448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:47.701571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7ebe-7412-49c4-ac45-2614fa696d12 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.709501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 188/366] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:47.710481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.742677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.743405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.743815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.744052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.746061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:47.747623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:47.754529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 179/367] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:47.755908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.788968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:47.792434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:47.832579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57f7a8-d5ae-4934-899d-a04a3f6d0e5e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.875899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.876593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.876878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.877233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.878649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:47.880988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:47.882878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:47.884001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab2ef1-23a9-4ae6-af5b-d5615c661c61 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:47.894386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 189/368] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:47.896169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:47.945173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:47.946187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:47.946639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:47.947164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:47.964526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:47.972649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:47.973605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:47.974433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 180/369] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:47.975129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.033479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e6785-7df1-4b76-a99d-322282f01484 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.067985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.073865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.074544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.074828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.075124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.076324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.077796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.079740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 190/370] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:48.080707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.102402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.117938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdea8-21bf-482f-afb5-0fff04394f6c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.155511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.156439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.156871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.157291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.195404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75fd8f-92aa-4091-9575-0aab7e6569b0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.230721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.232604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.234212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.235061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.235242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.235513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.236248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:48.236555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.238014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.247959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 191/371] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:48.248923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.320706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.333211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.349241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 181/372] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:48.350493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.365401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb427b-51ee-4b71-bd8a-8376f2f980e5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.403084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462849-e691-4df7-a92c-61777588260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.403706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462849-e691-4df7-a92c-61777588260a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.403993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462849-e691-4df7-a92c-61777588260a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.404272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462849-e691-4df7-a92c-61777588260a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.405077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462849-e691-4df7-a92c-61777588260a None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5a6a44082f1e4291b18bf0735dba8291) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:48.405391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462849-e691-4df7-a92c-61777588260a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.406993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462849-e691-4df7-a92c-61777588260a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.413377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 192/373] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:48.414293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462849-e691-4df7-a92c-61777588260a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.481885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf0df7-da54-49ce-af25-f6ba8121478b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.526294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462849-e691-4df7-a92c-61777588260a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.540373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.541086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.541378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.541659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.556585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.565639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.566308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.566590 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.566888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.568380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.569893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.572030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 193/374] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:48.579888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.616265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.633669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.635498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.699810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16b824-f1ec-4059-994d-be85f259523f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.746219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.746937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.747229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.747506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.748767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.749640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 182/375] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:48.751916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.753570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 194/376] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:48.756713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.764142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.839247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.858745 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:48.873811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee01515-140d-4d8e-a842-13481c49e6e5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.884803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e712e-63ba-4292-b391-78afb3ed8664 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:48.915319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2432b-2af6-4803-b82a-a2065f9f1c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.916107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2432b-2af6-4803-b82a-a2065f9f1c9a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.916460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2432b-2af6-4803-b82a-a2065f9f1c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.917018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2432b-2af6-4803-b82a-a2065f9f1c9a None admin] PATH_INFO: `/v3/projects/bb3fd2235b214e75887726b6b079e208` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.918142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2432b-2af6-4803-b82a-a2065f9f1c9a None admin] RBAC: Authorizing `identity:get_project(project_id=bb3fd2235b214e75887726b6b079e208)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.920734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2432b-2af6-4803-b82a-a2065f9f1c9a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.923060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 183/377] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/projects/bb3fd2235b214e75887726b6b079e208 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:48.924443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae1fb1-476b-4a39-8ce0-f76949c932cd None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.924816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae1fb1-476b-4a39-8ce0-f76949c932cd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.925143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae1fb1-476b-4a39-8ce0-f76949c932cd None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.926361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:48.926787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 184/378] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:48.927669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.928170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.928309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae1fb1-476b-4a39-8ce0-f76949c932cd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:48.928721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.929714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5a6a44082f1e4291b18bf0735dba8291) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:48.930183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:48.933169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca653b7b-8ef0-490c-b101-f73da80673b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:48.946942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 195/379] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:48.948804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:48.949327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:48.949806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:48.977810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:48.984284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:49.121619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 196/380] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:49.134076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae1fb1-476b-4a39-8ce0-f76949c932cd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:49.156379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:49.224202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:49.225233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:49.225689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:49.226199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] PATH_INFO: `/v3/groups/0353f07ac099470a80e5826a2fd2399f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:49.239562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] RBAC: Authorizing `identity:get_group(group_id=0353f07ac099470a80e5826a2fd2399f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:49.242130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:49.245642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 185/381] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:48 2022] GET /identity/v3/groups/0353f07ac099470a80e5826a2fd2399f => generated 229 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:49.251630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:49.279559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:49.299703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:49.370750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e179a-e723-4067-8c1a-c4c9193fec12 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:49.411360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:49.412004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:49.412680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:49.412966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:49.414073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] RBAC: Adding query filter params (group.id=0353f07ac099470a80e5826a2fd2399f, role.id=132431a386d641a8aaeba0496b601812, scope.project.id=bb3fd2235b214e75887726b6b079e208) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:49.414394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:49.415951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:49.416508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc19e64-10f8-4857-87cb-593a49d3db69 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:49.422440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 186/382] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:49 2022] GET /identity/v3/role_assignments?group.id=0353f07ac099470a80e5826a2fd2399f&scope.project.id=bb3fd2235b214e75887726b6b079e208&role.id=132431a386d641a8aaeba0496b601812 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:49.470983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:49.471977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:49.472409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:49.472840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:49.485831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:49.492769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:49.493948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:49.494949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:49.497449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 197/383] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:49.503233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:49.630449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:49.650750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d02e4c-c05b-4a75-a821-d516677984c9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:49.696372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:49.697308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:49.697735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:49.698149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:49.699055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:49.699530 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:49.701049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:49.712885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 187/384] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:49.718685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:49.832668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f23e3-74e6-4aba-97cb-4ab9b28d6a46 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:49.850594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:49.869184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:49.869859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:49.870147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:49.870423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:49.871686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:49.873156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:49.875171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 198/385] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:49 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:49.881533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:50.019843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c6b53-ecf1-4b0d-b8fc-13b65c00a884 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:50.062386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ead15-2ffa-444b-a5c0-be2d0f04ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:50.063135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ead15-2ffa-444b-a5c0-be2d0f04ce67 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.063423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ead15-2ffa-444b-a5c0-be2d0f04ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.063703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ead15-2ffa-444b-a5c0-be2d0f04ce67 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.064918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ead15-2ffa-444b-a5c0-be2d0f04ce67 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:50.066382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ead15-2ffa-444b-a5c0-be2d0f04ce67 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:50.068324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 188/386] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:49 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:50.079972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:50.165335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326acb5d-fb64-4721-9c92-009dfaa2ecba None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.165697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326acb5d-fb64-4721-9c92-009dfaa2ecba None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.165958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326acb5d-fb64-4721-9c92-009dfaa2ecba None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.167654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 189/387] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:50.178389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.178722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.178993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.215528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585156d-7883-4fba-9896-c1f3fae29261 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:50.221130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:50.222423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:50.256663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:50.257377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.257664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.257943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05/groups/a7388eb368554776b575c17b66d2c2ac/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.259314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:50.260063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] RBAC: Authorizing `identity:create_grant(project_id=5f2e064ab9374fbab081907969175e05, group_id=a7388eb368554776b575c17b66d2c2ac, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:50.261522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:50.280156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 199/388] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 00:59:50 2022] PUT /identity/v3/projects/5f2e064ab9374fbab081907969175e05/groups/a7388eb368554776b575c17b66d2c2ac/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:50.344395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 190/389] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:50.378688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:50.448773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:50.526611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d8c5-dcec-4f9e-9d62-109953b377cf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:50.564641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:50.565218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.565526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.565800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.577523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:50.584736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:50.586360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:50.589518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 200/390] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:50.597500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:50.737683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1883d-b500-428c-9acb-4cf661f678a1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:50.788631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:50.789305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.789595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.789880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.790682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:50.791062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:50.792518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:50.801817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 191/391] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:50.809717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:50.955820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b67ca-77c6-479d-a928-fea23a0169ca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:50.958636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:50.997173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:50.997847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:50.998131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:50.998410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:50.999615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:51.001068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:51.006982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 201/392] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:50 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:51.009093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:51.019295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:51.121299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4a3a3-b7fe-4b27-8b4e-c927650a95e9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:51.162813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46a923-14da-458a-bb35-1b25201b8cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:51.163528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46a923-14da-458a-bb35-1b25201b8cbd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.163813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46a923-14da-458a-bb35-1b25201b8cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.164087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46a923-14da-458a-bb35-1b25201b8cbd None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.167573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46a923-14da-458a-bb35-1b25201b8cbd None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:51.169064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46a923-14da-458a-bb35-1b25201b8cbd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:51.171198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 192/393] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:51 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:51.177029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:51.289924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c27c25-e78e-4679-ae32-117d7f9e2911 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:51.325404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9741c-6bf0-442b-aae3-1a0af0e7de78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:51.326106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9741c-6bf0-442b-aae3-1a0af0e7de78 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.326391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9741c-6bf0-442b-aae3-1a0af0e7de78 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.326672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9741c-6bf0-442b-aae3-1a0af0e7de78 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05/groups/a7388eb368554776b575c17b66d2c2ac/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.328966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9741c-6bf0-442b-aae3-1a0af0e7de78 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f2e064ab9374fbab081907969175e05, group_id=a7388eb368554776b575c17b66d2c2ac, role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:51.330461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9741c-6bf0-442b-aae3-1a0af0e7de78 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:51.345297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 202/394] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 00:59:51 2022] PUT /identity/v3/projects/5f2e064ab9374fbab081907969175e05/groups/a7388eb368554776b575c17b66d2c2ac/roles/8fc2a014f1e24e08a63b265436f41eec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:51.484224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edbf49-fa10-4e36-b2a1-6853aed15ffb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.484573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edbf49-fa10-4e36-b2a1-6853aed15ffb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.484804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edbf49-fa10-4e36-b2a1-6853aed15ffb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.486497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 193/395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:51.492373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57403572-b6ac-4f53-899e-b3e70d5e8369 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.492884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57403572-b6ac-4f53-899e-b3e70d5e8369 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.492957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57403572-b6ac-4f53-899e-b3e70d5e8369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.514206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-57403572-b6ac-4f53-899e-b3e70d5e8369 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:51.515491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-57403572-b6ac-4f53-899e-b3e70d5e8369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:51.547037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:51.600435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 203/396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:51.628419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edbf49-fa10-4e36-b2a1-6853aed15ffb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:51.720366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:51.754155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edbf49-fa10-4e36-b2a1-6853aed15ffb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:51.796037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:51.796705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.796989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.797378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.810754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:51.815854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:51.816739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:51.817526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:51.818949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0c8c-ac37-4649-b3ba-657079383056 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.819231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 194/397] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:51.819454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0c8c-ac37-4649-b3ba-657079383056 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.819750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0c8c-ac37-4649-b3ba-657079383056 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.821637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 204/398] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:51.823376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:51.831971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:51.832350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:51.832701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:51.863975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:51.866285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:51.985731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 205/399] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:52.013869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd030-03c4-458a-ab0e-1036a5aba388 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.047712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.078124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.078788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.079101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.079377 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.080141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:52.080456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:52.081877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:52.083752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:52.098132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 195/400] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:52.103803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.204006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:52.222715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b79c41-61a6-4a86-a5db-1d4ade51f279 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.249140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10db7c-ee4e-4f03-a95c-75cae20c6a8a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.287388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.288420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.288929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.289452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.290950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:52.292589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:52.294510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 196/401] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:52.302923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.307539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.308478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.309665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.311235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.339931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:52.346594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:52.349598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:52.350492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:52.352947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 206/402] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:52.354191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.393614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:52.439480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3a818-b1a7-410a-895a-12cd2b7cb585 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.493266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f3376-5c25-4024-a499-a1afca489dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.494408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f3376-5c25-4024-a499-a1afca489dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.495345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f3376-5c25-4024-a499-a1afca489dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.495977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f3376-5c25-4024-a499-a1afca489dc1 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.497534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f3376-5c25-4024-a499-a1afca489dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:52.500308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f3376-5c25-4024-a499-a1afca489dc1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:52.503301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 197/403] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:52.505161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c993-a2bc-412e-899d-8ac002eea146 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.505629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c993-a2bc-412e-899d-8ac002eea146 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.506064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c993-a2bc-412e-899d-8ac002eea146 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.512234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 198/404] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:52.513173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288c993-a2bc-412e-899d-8ac002eea146 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.566591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06ad8-6a22-4ab4-b70f-0fa8d0a91dca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.605180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.605946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.606645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.606645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.607839 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:52.607839 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:52.609076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65cecb-982b-4767-a55f-cda3543eff0d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:52.620050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 207/405] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:52.621439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.621736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.622014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.642338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:52.643780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:52.645165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288c993-a2bc-412e-899d-8ac002eea146 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.655525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:52.687677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:52.698730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.699688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.700600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.701025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291/groups/a7388eb368554776b575c17b66d2c2ac/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.703885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a44082f1e4291b18bf0735dba8291, group_id=a7388eb368554776b575c17b66d2c2ac, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:52.705535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:52.728780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 199/406] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 00:59:52 2022] PUT /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291/groups/a7388eb368554776b575c17b66d2c2ac/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:52.730330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.739066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 208/407] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:52.755131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.873133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52601cac-1a36-413a-9ff5-e457ab1ce5ea None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.891544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d1ff-56f8-4464-8ee1-8d78db8bdcac None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:52.918716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.919561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.919941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.920311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.921632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:52.923315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:52.925305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 200/408] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:52.931555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.945294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:52.946763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:52.947402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:52.947957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:52.964058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:52.969430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:52.970398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:52.971197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:52.978310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 209/409] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:52.980265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:52.985754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:53.015987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:53.067783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6e624-a72f-4b21-b5b5-2f0eb13c3d45 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.105234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.106183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.106605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.107065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.108446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:53.110037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:53.112289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 201/410] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:53.119375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:53.158630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881df6b-65b0-499f-94cd-c400e481dbf3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.209840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.210494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.210776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.211081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.212419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:53.212419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:53.213848 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:53.224623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 210/411] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:53.230980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:53.276865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fb993-3b46-441e-9014-140b20e769f9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.312992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:53.329255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:53.339380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce19e53-5bd7-44d3-8e65-4bc9a79fb59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.340523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce19e53-5bd7-44d3-8e65-4bc9a79fb59f None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.340813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce19e53-5bd7-44d3-8e65-4bc9a79fb59f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.341417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce19e53-5bd7-44d3-8e65-4bc9a79fb59f None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291/groups/a7388eb368554776b575c17b66d2c2ac/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.345039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce19e53-5bd7-44d3-8e65-4bc9a79fb59f None admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a44082f1e4291b18bf0735dba8291, group_id=a7388eb368554776b575c17b66d2c2ac, role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:53.346817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce19e53-5bd7-44d3-8e65-4bc9a79fb59f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:53.359237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b2c10-08b7-419e-a0f3-a7fb0812e8f4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.366585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 202/412] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 00:59:53 2022] PUT /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291/groups/a7388eb368554776b575c17b66d2c2ac/roles/8fc2a014f1e24e08a63b265436f41eec => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:53.375298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456598e-52d7-4f66-a22d-7c0a07693e7b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.375646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456598e-52d7-4f66-a22d-7c0a07693e7b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.375930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456598e-52d7-4f66-a22d-7c0a07693e7b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.380540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 203/413] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:53.386660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.387086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.387285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.427118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:53.428315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:53.448196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.448196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.448196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.448196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] PATH_INFO: `/v3/groups/0353f07ac099470a80e5826a2fd2399f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.448727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] RBAC: Authorizing `identity:get_group(group_id=0353f07ac099470a80e5826a2fd2399f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:53.451960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:53.460545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 211/414] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:53 2022] GET /identity/v3/groups/0353f07ac099470a80e5826a2fd2399f => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:53.464080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:53.561092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 204/415] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:53.583889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:53.662581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51fe7f-631e-445d-880c-f29b163c5362 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.719834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.720853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.721294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.721718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] PATH_INFO: `/v3/projects/bb3fd2235b214e75887726b6b079e208` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.727920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] RBAC: Authorizing `identity:get_project(project_id=bb3fd2235b214e75887726b6b079e208)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:53.729432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:53.735772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 212/416] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:53 2022] GET /identity/v3/projects/bb3fd2235b214e75887726b6b079e208 => generated 344 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:53.744072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bb918-be2c-435f-8082-9df6bc0e2add None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.754096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:53.762751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:53.765395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:53.789009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.789950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.790258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.791069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.803370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:53.809768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:53.810703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:53.811529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:53.812126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 205/417] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:53.817356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:53.937901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574aba-e2eb-4eb7-8701-b0d8f8104778 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:53.977159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:53.978386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:53.978939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:53.979316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:53.980333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:53.980707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:53.982191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:53.993383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 206/418] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:53.998670 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:54.059294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243606-1adc-4457-94f0-0615ea41b062 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:54.063453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:54.099612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214652b1-b827-46fd-8b90-30f310395e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:54.100334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214652b1-b827-46fd-8b90-30f310395e24 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.100622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214652b1-b827-46fd-8b90-30f310395e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.100901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214652b1-b827-46fd-8b90-30f310395e24 None admin] PATH_INFO: `/v3/projects/bb3fd2235b214e75887726b6b079e208/groups/0353f07ac099470a80e5826a2fd2399f/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.103182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214652b1-b827-46fd-8b90-30f310395e24 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb3fd2235b214e75887726b6b079e208, group_id=0353f07ac099470a80e5826a2fd2399f, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:54.104743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214652b1-b827-46fd-8b90-30f310395e24 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:54.123895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 213/419] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jul 28 00:59:53 2022] PUT /identity/v3/projects/bb3fd2235b214e75887726b6b079e208/groups/0353f07ac099470a80e5826a2fd2399f/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 00:59:54.132842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045b470-151c-4a40-9cc1-c4fb21f34b07 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.133112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045b470-151c-4a40-9cc1-c4fb21f34b07 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.133532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045b470-151c-4a40-9cc1-c4fb21f34b07 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.136182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 214/420] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:54 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:54.141653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.141991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.142347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.172059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:54.173854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:54.184940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae8a32-ab3c-47b4-9a35-7cd082acb452 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:54.279458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:54.282425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:54.283322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.283613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.283891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.315341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 215/421] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:54.351683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:54.432144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:54.436263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:54.493290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f7ef9-b661-417a-8f02-5fb07c64ee79 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:54.522468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:54.595661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 207/422] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:54.600935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:54.601228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:54.602210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.602686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.603234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.643879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:54.649251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:54.650363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:54.651196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:54.658198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 216/423] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:54.659761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:54.716444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60007df9-ad1f-4a7f-8467-52d5d5c4e442 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:54.762369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:54.763088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.763365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.763646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.764878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:54.766344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:54.768759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 208/424] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:54 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:54.773999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:54.783980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:54.786001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:54.837812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc854c-2639-4285-b997-7e4434dfcb7b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:54.874428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:54.875016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.875304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.875579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.876315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:54.876621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:54.878006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:54.882870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78064ad-662f-42a2-8bdc-b2a0218cc558 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:54.887530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 217/425] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:54.893289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:54.921587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:54.922256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:54.922541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:54.922824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:54.924180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:54.925642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:54.927515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 209/426] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:54 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:54.935323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:55.089897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:55.091520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f9485-9b07-4743-844e-348ce3075246 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:55.126132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:55.140450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fb18-542d-4c20-8d80-7637fcd25cbe None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:55.172506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:55.173142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.173434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.173707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.211682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:55.212393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.212732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.213018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.213840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5f2e064ab9374fbab081907969175e05) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:55.214155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:55.216781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:55.228424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 210/427] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:54 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 611 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:55.295943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:55.297525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:55.437286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 218/428] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:55.443549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:55.576631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab5454-c4ef-4de0-bf88-186d86fa0a03 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:55.612124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a9258-16f6-451c-b60d-6c155759686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:55.612794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9258-16f6-451c-b60d-6c155759686a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.613080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9258-16f6-451c-b60d-6c155759686a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.613369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9258-16f6-451c-b60d-6c155759686a None admin] PATH_INFO: `/v3/projects/5f2e064ab9374fbab081907969175e05` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.614587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9258-16f6-451c-b60d-6c155759686a None admin] RBAC: Authorizing `identity:get_project(project_id=5f2e064ab9374fbab081907969175e05)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:55.616088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9258-16f6-451c-b60d-6c155759686a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:55.618241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 211/429] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:55 2022] GET /identity/v3/projects/5f2e064ab9374fbab081907969175e05 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:55.625060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:55.638141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32509565-ceea-47c0-b4bd-6a26e480a281 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.638455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32509565-ceea-47c0-b4bd-6a26e480a281 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.638709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32509565-ceea-47c0-b4bd-6a26e480a281 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.640273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 212/430] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 00:59:55.645707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:55.646681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.647201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.647355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.674945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:55.675698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:55.678100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:55.751653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aa521-8cf8-4354-9690-b02e40b1226c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:55.759659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 213/431] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 00:59:55.795603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:55.801852 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:55.802527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.802812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.803123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.804305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:55.805781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:55.807831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 219/432] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:55 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:55.813705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:55.917203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:55.925834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654308d6-b0de-4207-bce6-f8859d339e37 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:55.944550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd2816-47a4-42ac-afa5-b32575b0119c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:55.963980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0713f-378e-47a9-b34e-11a87c65a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:55.964363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0713f-378e-47a9-b34e-11a87c65a678 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.964813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0713f-378e-47a9-b34e-11a87c65a678 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.965094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0713f-378e-47a9-b34e-11a87c65a678 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.975473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:55.981220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0713f-378e-47a9-b34e-11a87c65a678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2fb0713f-378e-47a9-b34e-11a87c65a678 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:55.987665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:55.988552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 00:59:55.989830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 214/433] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:55.990747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ef4b-d88e-4235-aaad-4ec1eeb206c6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.990747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ef4b-d88e-4235-aaad-4ec1eeb206c6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.990900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ef4b-d88e-4235-aaad-4ec1eeb206c6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.992660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 215/434] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:55.995444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ef4b-d88e-4235-aaad-4ec1eeb206c6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:55.996827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:55.997562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:55.997937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:55.998245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:55.999113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5f2e064ab9374fbab081907969175e05) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:55.999437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:56.000897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec671e5-2538-4a00-b39b-5789923b6756 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:56.007949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 220/435] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:55 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5f2e064ab9374fbab081907969175e05&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:56.009249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.009249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.009577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.032562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:56.035515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:56.132271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 221/436] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:56.140434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ef4b-d88e-4235-aaad-4ec1eeb206c6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:56.158269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:56.193488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:56.194305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.194527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.194818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.195630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:56.195943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:56.197485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:56.207738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 216/437] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 00:59:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 00:59:56.213786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:56.253457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:56.294967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132b77b-24c5-4cf8-875c-d6b3a22a7c6c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:56.296706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:56.348721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:56.349403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.349688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.350871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.360698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429fe90-860f-4e80-91f2-a28fdce17a9b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:56.365724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:56.373908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:56.375171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 00:59:56.376015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:56.377945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 222/438] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 00:59:56.379348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:56.403386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cac17-2834-4ad6-a89a-87c3d446096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:56.404035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cac17-2834-4ad6-a89a-87c3d446096b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.404320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cac17-2834-4ad6-a89a-87c3d446096b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.404595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cac17-2834-4ad6-a89a-87c3d446096b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.515000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cac17-2834-4ad6-a89a-87c3d446096b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:56.516586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cac17-2834-4ad6-a89a-87c3d446096b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:56.536543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ade9f8-bed8-4b64-8e81-fe3e6e58d5f2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:56.575347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:56.575979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.576325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.576685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.577608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:56.578129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:56.579919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f22ca-6096-44bc-b8c3-c801091f8d66 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:56.582708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:56.585281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:56.591509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 223/439] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:56.593679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f17ad-ad03-478c-ac67-c9fda0639d7b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.594173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f17ad-ad03-478c-ac67-c9fda0639d7b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.594654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f17ad-ad03-478c-ac67-c9fda0639d7b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.598247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 224/440] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 00:59:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 00:59:56.599334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f17ad-ad03-478c-ac67-c9fda0639d7b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:56.650132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 217/441] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 00:59:56.652328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.652834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.653340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.682653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 00:59:56.684734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 00:59:56.711173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f17ad-ad03-478c-ac67-c9fda0639d7b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:56.747931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:56.748576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.748833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.749103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.770502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 218/442] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 00:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 00:59:56.771411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:56.818610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:56.821085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:56.822712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:56.895889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:56.906493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0be49-cdb2-4e44-99f2-0e238197afd3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:56.925824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 225/443] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:56.926694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:56.945258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:56.945925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:56.946210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:56.946490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:56.947799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:56.949233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:56.950995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 219/444] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:56.951764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.031946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.036129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584311-14e0-42b8-aaa9-18771236f4ce None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.068121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b44df4-54ed-4a81-a7a1-1bb743f661ee None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.073186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.073865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.074182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.074430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.086456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 00:59:57.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 00:59:57.092540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 00:59:57.093608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 226/445] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 00:59:57.094523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.108198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.108540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.108793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.109072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] PATH_INFO: `/v3/projects/5a6a44082f1e4291b18bf0735dba8291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.110283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a6a44082f1e4291b18bf0735dba8291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.111789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.113519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 220/446] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:56 2022] GET /identity/v3/projects/5a6a44082f1e4291b18bf0735dba8291 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.114318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.148266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.211935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7768d79-a517-42cc-a364-f71904aeabe1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.223928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4fef8-0509-4642-99c8-0b42140406c1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.247492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.248199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.248453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.248735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.249928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.251484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.253101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 227/447] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.253869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.264432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.265129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.265416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.265730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.266464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:57.266751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.268182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.276971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 221/448] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.277807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.287917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.363876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeca0eb-e566-4974-b0a1-179864f8306d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.368137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.380981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6c1f-87d6-46af-9f2f-d0e90de7e21c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.399592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.400249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.400539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.400827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] PATH_INFO: `/v3/groups/a7388eb368554776b575c17b66d2c2ac` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.402063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] RBAC: Authorizing `identity:get_group(group_id=a7388eb368554776b575c17b66d2c2ac)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.403553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.405175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 228/449] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/groups/a7388eb368554776b575c17b66d2c2ac => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.405948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.420382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.421020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.421309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.421587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.422395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=90d9b9386f5b4e0ebc5bba9c29b9d836, scope.project.id=5a6a44082f1e4291b18bf0735dba8291) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:57.422709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.424201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.433162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 222/450] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.433162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.526778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.537210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28780705-68c2-45be-a276-98dc250fecfa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.566957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0941-7c28-4328-9eb1-364275c605ed None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.577230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.577903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.578384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.578490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.610868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.613255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.613255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.613255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.613255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] RBAC: Adding query filter params (group.id=a7388eb368554776b575c17b66d2c2ac, role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=5a6a44082f1e4291b18bf0735dba8291) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:57.613255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.614594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.624980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 223/451] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/role_assignments?group.id=a7388eb368554776b575c17b66d2c2ac&scope.project.id=5a6a44082f1e4291b18bf0735dba8291&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.660322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.661763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.774821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.783707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 229/452] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 00:59:57.790236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:57.812281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:57.900342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f214a-144d-4f98-b464-526f057e8b0f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:57.936383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:57.936966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:57.937248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:57.937524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] PATH_INFO: `/v3/projects/bb3fd2235b214e75887726b6b079e208` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:57.938711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] RBAC: Authorizing `identity:get_project(project_id=bb3fd2235b214e75887726b6b079e208)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:57.940215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:57.941896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 224/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/projects/bb3fd2235b214e75887726b6b079e208 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:57.947831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:58.046078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314b342-7ce3-4a97-b40f-81f5116f54cb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:58.081413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789f596-6227-4d60-b0ec-69fe8c1a3ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:58.082063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789f596-6227-4d60-b0ec-69fe8c1a3ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:58.082329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789f596-6227-4d60-b0ec-69fe8c1a3ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:58.082625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789f596-6227-4d60-b0ec-69fe8c1a3ad0 None admin] PATH_INFO: `/v3/groups/0353f07ac099470a80e5826a2fd2399f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:58.083862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789f596-6227-4d60-b0ec-69fe8c1a3ad0 None admin] RBAC: Authorizing `identity:get_group(group_id=0353f07ac099470a80e5826a2fd2399f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:58.085305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789f596-6227-4d60-b0ec-69fe8c1a3ad0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:58.086923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 230/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jul 28 00:59:57 2022] GET /identity/v3/groups/0353f07ac099470a80e5826a2fd2399f => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 00:59:58.093143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 00:59:58.185670 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 00:59:58.202291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12572567-f526-4b0b-b239-3041b7254f30 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 00:59:58.237032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 00:59:58.237644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 00:59:58.237927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 00:59:58.238204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 00:59:58.239025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] RBAC: Adding query filter params (group.id=0353f07ac099470a80e5826a2fd2399f, role.id=132431a386d641a8aaeba0496b601812, scope.project.id=bb3fd2235b214e75887726b6b079e208) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 00:59:58.239339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 00:59:58.240759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a51ec-1f51-4e1f-8d13-bc271ad2c1c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 00:59:58.246726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 225/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jul 28 00:59:58 2022] GET /identity/v3/role_assignments?group.id=0353f07ac099470a80e5826a2fd2399f&scope.project.id=bb3fd2235b214e75887726b6b079e208&role.id=132431a386d641a8aaeba0496b601812 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:00.727341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2d7d8-743b-4316-a69c-66ff54d35a93 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:00.727922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2d7d8-743b-4316-a69c-66ff54d35a93 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:00.727922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2d7d8-743b-4316-a69c-66ff54d35a93 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:00.729217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 231/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:00.734505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6b4dd-da13-4c0b-8b0b-22cf5b69839f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:00.734925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6b4dd-da13-4c0b-8b0b-22cf5b69839f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:00.735126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6b4dd-da13-4c0b-8b0b-22cf5b69839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:00.758952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8de6b4dd-da13-4c0b-8b0b-22cf5b69839f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:00.760291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8de6b4dd-da13-4c0b-8b0b-22cf5b69839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:00.847845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 226/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:00.894215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2d7d8-743b-4316-a69c-66ff54d35a93 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:00.955065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e1b0d-f440-466b-a65b-9f1a35b2cd18 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:00.955353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e1b0d-f440-466b-a65b-9f1a35b2cd18 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:00.955633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e1b0d-f440-466b-a65b-9f1a35b2cd18 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:00.957388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 227/458] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:00.962284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b8d2f-10f1-4723-bf9a-f073d759cf86 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:00.962573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b8d2f-10f1-4723-bf9a-f073d759cf86 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:00.962869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b8d2f-10f1-4723-bf9a-f073d759cf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:00.983683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.001090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-416b8d2f-10f1-4723-bf9a-f073d759cf86 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:01.002926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-416b8d2f-10f1-4723-bf9a-f073d759cf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:01.028471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2d7d8-743b-4316-a69c-66ff54d35a93 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.077061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.077729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.078173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.078453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.090933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 228/459] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 28 01:00:01.091641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.092373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd284c-d1e9-4be9-822a-d6c3de012046 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.092664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd284c-d1e9-4be9-822a-d6c3de012046 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.092940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd284c-d1e9-4be9-822a-d6c3de012046 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.093117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.098578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 229/460] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:01.100618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.100906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.101186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:01.121718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:01.122625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:01.123505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:01.123505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:01.123505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:01.123505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:01.123505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 232/461] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:01.124204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.129513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:01.131381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:01.133597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.214859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 230/462] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:01.214859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.264990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b361d-8116-41ef-b966-10cfca818fd8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.281716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.304430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.305374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.305794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.306286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.317898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.319601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.319938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b327-ecdc-4c0b-bab2-f1439d67842d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:01.336750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:01.337993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:01.338948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:01.338948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:01.338948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:01.338948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:01.341567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 233/463] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:01.343109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.356791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.357547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.357834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.358112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.358825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:01.359157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.360557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.368135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.379325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 231/464] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:01.380634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.456007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334df1-cb28-4831-873c-2700e1b9223d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.489291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ff60-f23e-47f6-afa9-2bb812cb0566 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.502823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.503890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.504545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.505390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.521432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.524025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.527761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.528494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.528849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.529192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.530034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:01.530415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.531973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:01.541614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:01.542506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:01.543371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:01.543371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:01.543371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:01.543371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:01.546031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 234/465] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:01.546973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 232/466] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:01.547694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.552661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.568977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.627002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.660524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc05e-dbef-4ef9-b19b-f7e292ad714e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.681246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb4ae9-521e-4719-bda2-bd12cd27da68 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.698525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ee517-89e6-4f1d-9cb5-7065b243a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.699229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ee517-89e6-4f1d-9cb5-7065b243a3e6 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.699519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ee517-89e6-4f1d-9cb5-7065b243a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.699801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ee517-89e6-4f1d-9cb5-7065b243a3e6 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.700502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ee517-89e6-4f1d-9cb5-7065b243a3e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.702062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ee517-89e6-4f1d-9cb5-7065b243a3e6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.717527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.718229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.718518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.718795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.719539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:01.719854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.721236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.738082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 233/467] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:01.738082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.738780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 235/468] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:00:01 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:01.738780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c08bf-f68e-4aae-a428-f3b588207fe4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.738780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c08bf-f68e-4aae-a428-f3b588207fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.738780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c08bf-f68e-4aae-a428-f3b588207fe4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.738780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 236/469] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:01.744313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c08bf-f68e-4aae-a428-f3b588207fe4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:01.854394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.883074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85a657-eb5c-491d-960f-d34dc2d12290 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.891392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c08bf-f68e-4aae-a428-f3b588207fe4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:01.904648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:01.930054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6decb01-f907-41a7-b5e2-a3ea3ef88031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.930686 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6decb01-f907-41a7-b5e2-a3ea3ef88031 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.931116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6decb01-f907-41a7-b5e2-a3ea3ef88031 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.931669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6decb01-f907-41a7-b5e2-a3ea3ef88031 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.932124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6decb01-f907-41a7-b5e2-a3ea3ef88031 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.933610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6decb01-f907-41a7-b5e2-a3ea3ef88031 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.937292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:01.937941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.938254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.938532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.939342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:01.940793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:01.964604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 234/470] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:00:01 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:01.966266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893a797-1896-4fab-9fa2-59ff7db0c0f2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:01.966746 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893a797-1896-4fab-9fa2-59ff7db0c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:01.967218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893a797-1896-4fab-9fa2-59ff7db0c0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:01.973827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 237/471] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:00:01 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:01.992858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4893a797-1896-4fab-9fa2-59ff7db0c0f2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:01.994274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4893a797-1896-4fab-9fa2-59ff7db0c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:02.102951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 235/472] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:02.149312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:02.213992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a702f-f027-41f7-a60f-c1876783f162 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:02.214312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a702f-f027-41f7-a60f-c1876783f162 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:02.214559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a702f-f027-41f7-a60f-c1876783f162 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:02.218213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 236/473] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:02.224635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:02.224927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:02.225202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:02.255547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:02.257251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:02.282294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9742654-7d4e-43f5-aae8-3faa10c47eb8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:02.314258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:02.322618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:02.323348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:02.323641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:02.323929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:02.338065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:02.339907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:02.357678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:02.358597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:02.359454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:02.359454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:02.359454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:02.359454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.359671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 237/474] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:02.359740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 238/475] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:02.366122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:02.385394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:02.491111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:02.499765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741eec6f-df2c-415c-a9a5-028000e1e00c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:02.537581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:02.538334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:02.538631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:02.538942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:02.539701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:02.541185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:02.550721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 01:00:02.550721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 01:00:02.550721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [parameters: {'id': 'd6d4f68cbeba4d3d83faad430105949b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 01:00:02.550721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 28 01:00:02.587517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 01:00:02.588526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 01:00:02.589413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 01:00:02.590258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 01:00:02.593053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 01:00:02.595148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'id': 'd6d4f68cbeba4d3d83faad430105949b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:02.603434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 01:00:02.604954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.606044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 239/476] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 01:00:02 2022] POST /identity/v3/roles => generated 144 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:02.606044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:02.650119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e34d7-4be2-423d-9f07-227f3b843a4f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:02.703652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:02.727281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:02.727892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:02.728175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:02.728453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:02.729182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:02.729581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:02.731052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:02.765880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 238/477] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:02.767164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:02.833655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1faa77-3900-467e-ba10-5f1380ff8e53 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:02.878926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:02.883075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:02.883296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:02.883577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:02.901403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:02.907110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:02.907986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:02.908766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 240/478] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:02.916030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:02.977257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:02.989809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642b5dd-2024-42c3-834f-be75aeed57ae None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:03.055052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b69d-cb60-43cc-874d-d3dddbec6875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:03.055689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b69d-cb60-43cc-874d-d3dddbec6875 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:03.056121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b69d-cb60-43cc-874d-d3dddbec6875 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:03.056395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b69d-cb60-43cc-874d-d3dddbec6875 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:03.057089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b69d-cb60-43cc-874d-d3dddbec6875 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:03.058521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b69d-cb60-43cc-874d-d3dddbec6875 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:03.078739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb180-46ff-4e90-9db6-b7572dc556b9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:03.088347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:03.109573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 239/479] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:00:02 2022] POST /identity/v3/users => generated 249 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:00:03.119454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:03.120166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:03.120451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:03.120725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:03.121615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:03.122042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:03.123625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcac740-8ecb-4d61-b678-81038aecd334 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:03.134931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 241/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:04.342556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464af84-364c-4732-ac76-0a28d54f1f38 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.343136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464af84-364c-4732-ac76-0a28d54f1f38 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.343136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464af84-364c-4732-ac76-0a28d54f1f38 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.344744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 240/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:04.350759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.351660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.352127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.371257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:04.372819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:04.406642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba74b3-b310-46d5-bee3-a1499145bdab None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.406642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba74b3-b310-46d5-bee3-a1499145bdab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.407087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba74b3-b310-46d5-bee3-a1499145bdab None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.409735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 241/482] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:04.415972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d1853-b4ec-4771-ad4a-8f66bf13750c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.416340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d1853-b4ec-4771-ad4a-8f66bf13750c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.416624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d1853-b4ec-4771-ad4a-8f66bf13750c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.456136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 242/483] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 01:00:04.462917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7c5d1853-b4ec-4771-ad4a-8f66bf13750c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:04.464140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7c5d1853-b4ec-4771-ad4a-8f66bf13750c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:04.483991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:04.593227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:04.596696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 242/484] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:04.598675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c755-14a6-4af5-81a5-45bebc1490ae None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.599306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c755-14a6-4af5-81a5-45bebc1490ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.599772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c755-14a6-4af5-81a5-45bebc1490ae None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.601663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 243/485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:04.610873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.610873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.611191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a385984-5de7-4f08-b170-abc260c81841 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:04.611294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.634443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:04.635776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:04.660339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:04.660999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.661284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.661558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.672491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:04.677436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:04.678511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:04.679557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 243/486] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:04.680355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:04.723447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 244/487] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 01:00:04.724722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:04.737157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:04.799633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab5839-32e9-4df2-a4a1-71cc72684e25 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:04.836814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:04.837692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.837957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.838245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.850053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:04.851075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ad47d-c453-45a3-830f-d9abb56e3a5f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:04.855258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:04.856145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:04.856903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:04.861614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 244/488] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:04.862943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:04.887362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:04.893300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:04.894334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:04.894945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:04.895433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:04.896431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:04.896994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:04.898884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:04.908677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 245/489] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:04.914574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:04.986719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379fe1-95f6-4aa5-8921-097a398bc323 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:04.990569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.025784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7730f-6436-4539-8e5e-00ae9c7c9d13 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.037145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.038272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.038749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.039261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.051489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:05.058097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:05.059503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:05.060668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:05.062882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.063542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.063824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.064096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.064523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 245/490] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:05.065128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:05.065436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.065959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.066819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.081594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 246/491] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:05.081594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.163865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.189962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54755c-6a97-4c55-a9bc-135190e4ddb3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.207303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e87b1-daeb-428c-aab3-55e4e338fe60 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.229516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.230556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.231651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.232162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.246085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.246725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.247056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.247331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.248061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:05.248367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.248799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.249802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.251050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.251812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.262009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 247/492] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:05.262986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:05.278727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:05.279678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:05.280609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:05.280609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:05.280609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:05.280609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:05.283517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 246/493] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:05.284592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.399053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486a290-f639-43e3-b9ab-0ec44b946625 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.406774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e88494-3d74-471c-9218-3cdc3747fc99 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.439094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.455479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.455479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.455479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.455479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.455939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.455939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.455939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.456193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.468392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.469332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.469974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.471086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:05.490806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:05.491751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:05.492566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:05.492566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:05.492566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:05.492566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:05.492566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 248/494] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:05.493330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:05.493961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:05.494737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:05.495846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:05.495846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:05.495846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:05.495846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:05.497661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 247/495] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:05.503126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.548309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.605665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b89919-cbe6-4156-8782-349d6cd0b7ec None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.640823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b01495-716b-4a1a-85a2-ab6c0cae83bc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.644481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.645131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.645413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.645754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.646435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:05.646741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.648214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.661172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 249/496] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:05.661791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.697923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.698935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.699327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.699755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.700902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:05.701404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.703886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4109a-1f27-422e-b1b1-54e2c28d2cfd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.724134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 248/497] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:05.726254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e6c7-062f-4180-9969-b66d875d6f75 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.726724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e6c7-062f-4180-9969-b66d875d6f75 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.727222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e6c7-062f-4180-9969-b66d875d6f75 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.729437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 249/498] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:05.730647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031e6c7-062f-4180-9969-b66d875d6f75 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.772566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe973-5099-45b2-905e-2bf729c90cda None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.784042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.794173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:05.813880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.814522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.814806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.815129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.815842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:05.816153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.817555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff78c49-0dd2-49f3-a5bf-009a5aae15f2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.829039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 250/499] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:05.830307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae5f7-a131-49b1-8f20-fd7b9b0c236c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.830599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae5f7-a131-49b1-8f20-fd7b9b0c236c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.830903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae5f7-a131-49b1-8f20-fd7b9b0c236c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.832362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 251/500] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:05.832848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae5f7-a131-49b1-8f20-fd7b9b0c236c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:05.855908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031e6c7-062f-4180-9969-b66d875d6f75 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.904656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.905344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.905629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.905905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.914406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.915922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-061b7f8a-bbf3-473d-a5e6-23c5c69e4cd3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:05.930457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:05.931692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:05.932577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 250/501] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:05.933523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e778e-0ca8-4c67-8cf2-d3a23180de64 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.933808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e778e-0ca8-4c67-8cf2-d3a23180de64 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.934192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e778e-0ca8-4c67-8cf2-d3a23180de64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.945030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae5f7-a131-49b1-8f20-fd7b9b0c236c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:05.953280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b52e778e-0ca8-4c67-8cf2-d3a23180de64 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:05.954406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b52e778e-0ca8-4c67-8cf2-d3a23180de64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:05.981558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:05.982208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:05.982492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:05.982767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:05.990075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:05.991546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:06.005174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:06.006148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 252/502] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:06.007147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.025104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 251/503] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:06.026369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.026866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.026995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.039355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.051236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:06.052446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:06.079115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.113458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8bd3-9a5c-4f04-8158-63046121bbc7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.121435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 252/504] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:06.122280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.149108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.149798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.150084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.150355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.158373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.159860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:06.172578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:06.173542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:06.174415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 253/505] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:06.174987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.227755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023ceff-1926-4d49-901d-7b44a83cd3a3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.262182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.263104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.263751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.264038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.264362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.265242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:06.265564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.267062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.275082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f858-0245-49ca-817c-2429a58a2a74 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.303599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 253/506] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:06.304469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.314090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.314714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.315019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.315287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.316109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:06.316412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.317858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.326470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.355786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 254/507] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:06.356743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.407969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922602-8859-4e99-92f8-ba75263812a5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.453731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.454409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.454690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.455005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.465955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.467467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.472421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e678dce-fd47-41e6-bb99-6dfbc816e1a1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:06.487769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:06.488720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:06.489561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:06.489561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:06.489561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:06.489561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:06.489561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 254/508] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:06.489825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.510586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.511269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.511551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.511825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.522165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:06.527355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:06.528397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:06.529172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 255/509] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:06.529816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.539039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.564799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.596176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14d13-e4b8-436c-bfdb-8625ccf21788 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.635440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.636074 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.636359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.636636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.637448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:06.637761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.637881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5c068-8f75-4d3c-b431-7b735c52aa2d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.639245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.657500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 255/510] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:06.658400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.677754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.678379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.678749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.678975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.749198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.750635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.760173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.764351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677a0b1-c679-488c-8214-7ff08a64c469 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.794626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.801438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.802058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.802344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.802622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.852056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 256/511] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:06.853192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.871086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.872618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:06.955031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c0107-7215-440a-b6e3-449f90b6a325 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:06.962353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:06.972190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 256/512] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:06.973371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:06.994103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:06.994751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:06.995078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:06.995353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:06.996059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:06.996369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:06.997766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.003631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.010184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 257/513] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:07.011066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.079897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f56414-cb8e-43c3-8b9a-62ab54c746fa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.112657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533ee5-8ef1-4789-8a22-102de154e45d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.118198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.118972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.119257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.119540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.120257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:07.120573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.121951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.130956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 257/514] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:07.131796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.150990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.151627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.151913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.152196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.195688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.231201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.234810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.236403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.241046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76ece1-f329-4af3-8ce9-46bf624fe8d0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.276411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.277008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.277291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.277565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.288664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.290167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:07.305399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:07.306466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:07.307328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:07.307328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:07.307328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.307328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:07.307328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 258/515] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:07.307593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.338121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 258/516] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:07.339009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.406687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.414011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454e630-165d-4d79-b2e3-30d9b961dc7f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.443628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86146fd4-95f5-4e44-8fe8-943d862000d3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.457295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.458063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.458406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.458687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.470094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.471764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.476145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.491412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.492055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.492330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.492672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:07.493230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:07.494090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:07.494961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:07.494961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:07.494961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.494961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:07.495165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.495239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 259/517] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:07.495304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.495376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.518874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 259/518] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:00:07 2022] POST /identity/v3/users => generated 247 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:07.519763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.615770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94681ca3-9fb5-4049-84ed-b70356f71b63 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.658110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797dcc-4a85-4bf5-b027-7fbc9bf2a4b0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.662187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.662902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.663191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.663466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.678664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.680186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.688764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:07.701012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:07.702036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:07.702993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:07.702993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:07.702993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.702993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:07.702993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 260/519] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:07.703264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.706387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.708292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.708732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.709864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.710013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:07.710226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.711657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.727149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 260/520] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:07.727922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.769622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:07.838597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15569e0-5fdb-4947-8f80-b1bb9df757df None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.848267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff4d8c-fd3e-4fb9-b08b-c41a34dc6477 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:07.875557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.876131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.876412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.876711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.888287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:07.888838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:07.889126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:07.889396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:07.889733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.890093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:07.890396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:07.891514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.891781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:07.904644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 261/521] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:07.906633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:07.907840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:07.909422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:07.911158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:07.911158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:07.911158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:07.911158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:07.913541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 261/522] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:07.914471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:07.963969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.028066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788af71-b6e7-4cf0-91a1-c70d5399ffbf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.039719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.042696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ecf13-4048-46d2-a498-6a50894e3b0a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.069226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.069852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.070181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.070454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.071222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:08.071577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.072970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.079381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.080042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.080321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.080596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.094869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 262/523] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:08.095168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.095329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.095502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:08.102709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:08.103614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:08.104441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 262/524] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:08.105253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.222990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871375f-1a15-427f-91d4-96c21d8b8868 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.224977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.228523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84bdf7-352e-4757-95a8-e4fde68d96a9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.263013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.263678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.263961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.264239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.269864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.271075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.271488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.271904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.272050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.272825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:08.273248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.273486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.275114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:08.286495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:08.287453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 263/525] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:08.288906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.293502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 263/526] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:08.294556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.320212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.410713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc0d1-b5bc-4beb-96e9-672599085178 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.442673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a7b5d-a6eb-4b5a-9719-dc5cdf4094f0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.453133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.453826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.454113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.454420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.464313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.465785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:08.479897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:08.480871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:08.481923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 264/527] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:08.482487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.501260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.501919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.502209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.502494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.503323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:08.503627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.505496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.517997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.527861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 264/528] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:08.528709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.619631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4c7f5-164d-43ee-8493-d0d6a91976d4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.621744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.646021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214ce0e-db16-4a2b-998e-061de147debc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.659581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.660220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.660511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.660788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.661676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:08.661993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.663480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.687454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 265/529] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:08.687454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.690818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.691534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.692680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.692680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.700373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.701920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:08.714062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:08.715149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:08.716187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:08.720084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 265/530] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:08.721007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.779344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.799866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e683a-de4c-49c3-b345-975e1ce458a9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.827114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f220a7-46c0-4e0d-9522-97973d443fec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:08.837066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.837681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.837961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.838237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.839066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:08.839384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.840829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.862969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 266/531] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:08.862969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.868902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:08.869543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:08.869839 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:08.870087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:08.870958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8f2f77d4f4c44890ac1ab88ff66100bd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:08.871265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:08.872779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:08.879044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 266/532] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/role_assignments?user.id=8f2f77d4f4c44890ac1ab88ff66100bd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:08.880258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:08.884477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:08.996990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e424718-6a5d-47bd-9eea-9c9afb7843f0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:09.022320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551289f0-9b08-4c33-bb50-bbfbdbb38b5b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:09.043291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:09.043947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.044238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.044519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.045662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=a0c7dc51159c4fcd8ae520f82ef7ef61) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:09.045981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:09.047523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:09.054939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 267/533] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/role_assignments?user.id=a0c7dc51159c4fcd8ae520f82ef7ef61&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:09.056006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:09.067257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:09.083779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:09.084471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.085214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.085502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.086309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:09.086627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:09.088288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:09.120305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 267/534] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:09.133439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:09.251004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:09.266326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca6c0f-fb78-4482-b193-c0ce9f6acccd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:09.310367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273d0e5-b2f2-44aa-bf5f-9adbf1fe584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:09.311689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d0e5-b2f2-44aa-bf5f-9adbf1fe584f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.312304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d0e5-b2f2-44aa-bf5f-9adbf1fe584f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.312778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d0e5-b2f2-44aa-bf5f-9adbf1fe584f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.317641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1e79-b91b-4896-a53b-4b815719f89e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:09.383222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:09.383922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.384215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.384498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.385361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=6146fcd207164df18b9c08e8fcc1adb6) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:09.385683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:09.387391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:09.394098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 268/535] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:09 2022] GET /identity/v3/role_assignments?user.id=6146fcd207164df18b9c08e8fcc1adb6&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:09.394487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273d0e5-b2f2-44aa-bf5f-9adbf1fe584f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:09.402230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273d0e5-b2f2-44aa-bf5f-9adbf1fe584f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:09.405236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:09.547618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 268/536] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:09.554580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:09.632998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd908-536b-4f45-9c92-6b6750ea7cd7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.633347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd908-536b-4f45-9c92-6b6750ea7cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.633729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd908-536b-4f45-9c92-6b6750ea7cd7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.635590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 269/537] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:09.641053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.641473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.641853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.661464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:09.662808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:09.720911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f090c5-e738-46e3-848e-9c49c7d13c46 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:09.741049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 270/538] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 01:00:09.759499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:09.768688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:09.769382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.769694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.770027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.780984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:09.781978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:09.783640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:09.799202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:09.800203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:09.801020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:09.801020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:09.801020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:09.801020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:09.805103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 269/539] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:09.806438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:09.901387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:09.914986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf201a-a361-4595-a265-09ec6b27dec0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:09.952519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:09.953222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:09.953517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:09.953799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:09.954517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:09.954849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:09.956407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:09.974066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 270/540] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:09.975397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:10.025199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:10.088696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cea31-376a-4f80-9e21-b9afa171e26b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.104998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8e6da-12e6-4caa-abca-3feab4515eaa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.162219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.162808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.163118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.163394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.175734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.176402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.176825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.177115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.191906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:10.196779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:10.197682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:10.200308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:10.201221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:10.202037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:10.202660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 271/541] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:10.211387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:10.220634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:10.222656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:10.224672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:10.234131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 271/542] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:10.244589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:10.359759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:10.378910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5e4f4-472c-496a-8887-b90dfd2b2b30 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.436682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:10.441190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.441847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.442128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.442406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.443204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:10.443502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:10.448539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9e7d8-4149-4d9d-95e9-9b89d261b563 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.448730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:10.468225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 272/543] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:10.472217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:10.523095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.525245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.525912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.527597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.528622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:10.529355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:10.531968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:10.549546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 272/544] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:10.556924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:10.633220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c675af2-8f87-4e7f-b25d-6c1bcaff5c68 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.674818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.675518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.675794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.676075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.697382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:10.698771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:10.731521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:10.732441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:10.733264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:10.733264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:10.733264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:10.733264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:10.735899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 273/545] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:10.744257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:10.756777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f82ef-04f0-44b8-ae7c-8c754d822627 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.769248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:10.796954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.797597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.797882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.798160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.799010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=5fda0e3ee3624ac19de85bb3db30e2dd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:10.807033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:10.808771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:10.819157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 273/546] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:10 2022] GET /identity/v3/role_assignments?user.id=5fda0e3ee3624ac19de85bb3db30e2dd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:10.894606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b756b7-420d-4a51-87ed-e99714554d54 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:10.946795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:10.947984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:10.948487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:10.948994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:10.949982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:10.950515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:10.952701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef4fb-7abf-4c49-874d-7d564ab970b1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:10.972936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 274/547] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:10.978867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:11.019310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:11.122035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9ba96-e95e-48b5-a76e-3f229ddad40d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:11.160309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60ab9b-0a7f-4870-8781-8c2f770afa85 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.160647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60ab9b-0a7f-4870-8781-8c2f770afa85 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.160877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60ab9b-0a7f-4870-8781-8c2f770afa85 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.162519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 275/548] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:11.168070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.168349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.168634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.181664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:11.182721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.182721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.183033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.196280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:11.197570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:11.200362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:11.201803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:11.222280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:11.224613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:11.226435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 274/549] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:11.236000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:11.303652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:11.312920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 276/550] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 01:00:11.337956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:11.435472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:11.463224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492e9b3-a9eb-469e-a6f6-81e22c7b57f1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:11.505837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:11.506482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.506764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.507066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.508469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:11.508852 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:11.510320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:11.538079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 275/551] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 01:00:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:11.547934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:11.568873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832208c-16e4-4fa7-a78e-0baf7bf6816c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:11.645757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:11.646423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.646785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.647095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.653432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b3f6e-e243-498b-a3e4-ac88d8b8b90a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:11.670097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:11.675183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:11.677433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:11.679359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 277/552] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:11.687930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:11.695629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17414e-a78b-4c68-a3d9-cd38a009fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:11.696325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17414e-a78b-4c68-a3d9-cd38a009fa32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.696610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17414e-a78b-4c68-a3d9-cd38a009fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.696886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17414e-a78b-4c68-a3d9-cd38a009fa32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.712368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:11.768659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17414e-a78b-4c68-a3d9-cd38a009fa32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:11.770178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17414e-a78b-4c68-a3d9-cd38a009fa32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:11.818058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:11.873237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec837e2-0c07-4d9f-b185-ef0541928d37 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:11.873652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 276/553] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:11.879568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2c916-fd5d-4cf1-af17-564575ffeb42 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.880284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2c916-fd5d-4cf1-af17-564575ffeb42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.880573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2c916-fd5d-4cf1-af17-564575ffeb42 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.882239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 277/554] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:11.883031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2c916-fd5d-4cf1-af17-564575ffeb42 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:11.914058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:11.914689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.915133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.915412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.916141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:11.916451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:11.917948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4d365-4df1-42da-95c2-5d29475076a4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:11.931474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 278/555] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:11.932850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:11.933167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:11.933452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:11.935207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:11.953881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:11.955201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:11.993421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2c916-fd5d-4cf1-af17-564575ffeb42 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.030710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.031380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.031662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.031930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.043199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.044919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.045400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 279/556] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:12.046276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:12.078093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.080213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.082205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:12.082205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:12.082205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:12.082205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:12.087094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 278/557] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.087094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.116738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.179625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214abd6-ef2e-4e43-b955-3ab452ebd7fb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.189295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.198051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c227ba-3bbb-41ae-8d29-0881f9cb4995 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.220545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.221307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.221603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.221999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.233699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.233860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.234425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.234709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.235079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.235287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.246188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-da4140d0-52fa-4849-bdf1-5420a48aae68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:12.251185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.252504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.253400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:12.253400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:12.253400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:12.253400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:12.253619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.254471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 279/558] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.255698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.257517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 280/559] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.258943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc8ae5-f19f-42a6-bab7-22b8db2b2d7b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.259237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc8ae5-f19f-42a6-bab7-22b8db2b2d7b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.259519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc8ae5-f19f-42a6-bab7-22b8db2b2d7b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.260969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 281/560] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:12.261525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc8ae5-f19f-42a6-bab7-22b8db2b2d7b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.372059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc8ae5-f19f-42a6-bab7-22b8db2b2d7b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.373742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.375230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca77cf-778c-4e73-b22e-369ce6b4e942 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.410489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.411214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.411537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.411764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.412183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.412461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:12.412769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.412872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.413105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.413385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.414094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:12.414192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.414406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.415862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5eb50e-104e-4973-89e8-d13d01efb6b8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.434883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 280/561] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:12.435139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 282/562] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:12.435240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.435337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.435337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.435337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.454719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:12.455878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.455992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:12.535007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 283/563] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:12.536572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.548073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0582f3-3032-4fcc-a112-da353a2bfeda None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.584350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.584995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.585277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.585557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.586294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:12.586606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.588105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.601433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 281/564] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:12.601433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.647176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.657925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068c232-86eb-4c1d-b6d1-95ae04189297 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.694692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.705796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.707257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.707775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.708286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.716387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.717957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.719200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd788a78-3985-4f3b-96ea-769af5fdbea2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:12.729818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.730792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 284/565] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.731812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.760496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.761155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.761440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.761735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.769654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.771184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:12.783241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.784271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 282/566] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.785221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.847701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3e521-85a7-44c6-883e-a2593fe06564 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.885058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.885821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.886110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.886389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.896736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd4d4-f366-49b8-bfe9-9183098511a4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:12.897079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:12.902656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.903608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:12.904508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:12.908723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 285/567] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.909617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:12.926399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.937479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:12.938145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:12.938426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:12.938699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:12.941751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:12.951590 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:12.953251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:12.971873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:12.972844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:12.974488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:12.974488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:12.974488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:12.974488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:12.974488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 283/568] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:12.975051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.037509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642af8c-2464-449a-a1b7-f785bbe993af None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.075360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.076002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.076290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.076568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.077526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:13.077749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.079223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.097543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 286/569] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:13.097543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.098977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f407694-a1a8-4354-b2bf-19f03ae5123b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.134313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.135017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.135306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.135586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.136408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:13.136723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.138183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.161599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 284/570] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:00:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:13.161599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.173577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:13.205181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0cb79-2755-42b4-afcf-38a121b3d188 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.210757 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:13.240957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.241658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.241869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.242148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.242914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:13.243444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.244743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c016860-09ff-4a2c-adcb-17adb4112f3d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.253165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 287/571] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:13.254467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2173f-70e9-4e34-8c03-8e38b9ebfc1d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.255182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2173f-70e9-4e34-8c03-8e38b9ebfc1d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.255460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2173f-70e9-4e34-8c03-8e38b9ebfc1d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.256682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 288/572] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:13.257374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2173f-70e9-4e34-8c03-8e38b9ebfc1d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.271114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a593-ce17-416e-bbf1-33e8c4f7026f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.307023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.307653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.307940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.308218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.308923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:13.309228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.310614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.322004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 285/573] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:13.322856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.368989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2173f-70e9-4e34-8c03-8e38b9ebfc1d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.406130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.406779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.407104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.407384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.408197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=b4423ec5dc2f465086495db19afde175) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:13.408530 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.410042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.412277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:13.417945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 289/574] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/role_assignments?user.id=b4423ec5dc2f465086495db19afde175&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:13.419777 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.441113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b32b7-9351-4d01-beee-3bb801dffae6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.451935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:13.490350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.491374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.491518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.491879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.505339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.507254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cffcbd43-11b5-4b6a-888b-24ae4d8d0a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:13.527194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:13.528671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 286/575] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.530197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.554247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880f75-d482-4ecc-9ea8-a64737cbe6c3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.556215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:13.558712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:13.596773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.597561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.597885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.598167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.610419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.611970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:13.629186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:13.630781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:13.632140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:13.632140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:13.632140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:13.632140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:13.632140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 290/576] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:13.632140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.653330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 287/577] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:13.660099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.689374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:13.757096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec70d-6be1-433b-9475-26afadbad137 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.770174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:13.787910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42295810-bacd-4891-a9a2-d72af6d9ee7e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.809023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.810373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.811125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.811876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.827167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:13.827624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:13.828168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:13.828168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:13.829049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:13.829302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.829687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:13.830917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.832828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:13.848391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 288/578] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:13.848391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:13.860448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:13.861646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 01:00:13.862812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:13.864541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 291/579] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:13.865764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:13.972769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19815dc-7387-4491-be44-dfadab9c71ec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:13.994401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93950b79-3bf9-4c28-bacc-0dccee6253ef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.006339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.010764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.011435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.011722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.011996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.012700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:14.013005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.014709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.027566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 289/580] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:14.028659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.040534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.042781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.043815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.044107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.044378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.060800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:14.068173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:14.069200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:14.070012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:14.071366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 292/581] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:14.074885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.150355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3f0bb-5dbf-4916-89b0-f3b912ac2c59 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.188894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.189563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.189846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.190119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.198981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.200460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.209425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c83e60-46c7-4c95-8715-3bb3f00a276f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:14.212501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:14.213596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 290/582] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:14.214473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.248892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.249473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.249756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.250039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.250863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:14.251166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.252957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.276474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 293/583] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:14.276474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.292564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.306735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.328323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727b705-7358-4e28-bd11-b67c022afb93 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.366815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.367921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.368268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.368639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.378011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.379514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:14.391686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:14.392668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:14.393594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 291/584] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:14.394172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.400284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89092491-4d8e-44a9-92d1-4cb9e20e512b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.437211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.437843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.438124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.438400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.439176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:14.439483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.441044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.454727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 294/585] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:14.454727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.520999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3aa41-3ed4-496a-9790-ece7610d7cb3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.549080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.559179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.560106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.560246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.560489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.561259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:14.561572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.563032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.563932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.573684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964bb0e-d76f-4cdc-9442-7d1e73e3676f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.583657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 292/586] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:14.583657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.613154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.613819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.614109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.614391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.625756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.627322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:14.642660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:14.643672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:14.644546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:14.644546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:14.644546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:14.644546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:14.644546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 295/587] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:14.644840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.698058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8b15b-6aa3-4c74-b27c-c8152414a169 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.735784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.736440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.736723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.737008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.737813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:14.738122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.739706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.758396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91eebc-1726-442d-bb95-230859d688be None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.764298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 293/588] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:14.765001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.799218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.799816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.800205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.800389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.812278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.813754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.815599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.819150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:14.828783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:14.829675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:14.835715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:14.835715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:14.835715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:14.835715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:14.835715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 296/589] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:14.835715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.883096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bc30b-e9ce-4b2a-b421-6d9d29d41318 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.918520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.919251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.919540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.919822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8f2f77d4f4c44890ac1ab88ff66100bd/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.934872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=8f2f77d4f4c44890ac1ab88ff66100bd, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.936337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:14.943740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f8ef8-d8bf-44d1-b226-60dcd060469e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:14.952347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 294/590] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:14 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8f2f77d4f4c44890ac1ab88ff66100bd/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:14.953515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:14.984780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:14.985440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:14.985728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:14.986005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:14.986742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:14.987133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:14.988595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.008424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 297/591] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:15.009534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.077135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.110537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.120476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1146c-a29c-4f40-94aa-a18c7707cbca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.137576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6284b-6a8e-40ab-ba3e-2f3d8c36727c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.181002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.181668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.181957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.182075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.182261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.182716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.183041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.183318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.184095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:15.184508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.185990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.195086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.197477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.199603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 298/592] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:15.200630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:15.214733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:15.215704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:15.216562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:15.216562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:15.216562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:15.216562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:15.217195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 295/593] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:15.218310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.332927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625a2ba-656a-4ba8-97dc-ae666c660af4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.340704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a165af-8892-4634-b173-455ef47ea855 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.359332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.371221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.372046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.372410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.372810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.381759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.383412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.386930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.387772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.388207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.388663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:15.395914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:15.397076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 01:00:15.398245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:15.399662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.401281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.404724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 296/594] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:15.405733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:15.415485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:15.416607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:15.417703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:15.420368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 299/595] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:15.421376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.426123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.540863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e294861-b0ba-4117-96cc-27f61ea68fe8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.541076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c9029-e852-4632-9805-3f13850b1d62 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.578247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.578905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.579190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.579473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.580165 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:15.580469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.581646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.581862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.582680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.583433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.583933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.584973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:15.585485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.587423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30360417-c1a5-462e-bf5b-28baed4e110a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.594339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 297/596] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:15.595221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.612069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 300/597] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:15.613419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2d276-37a8-4ccf-9aa7-ebedfc6439e5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.613713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2d276-37a8-4ccf-9aa7-ebedfc6439e5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.613983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2d276-37a8-4ccf-9aa7-ebedfc6439e5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.615508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 301/598] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:15.619895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2d276-37a8-4ccf-9aa7-ebedfc6439e5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.655680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.681303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.717454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843ad37-f225-4ec9-b956-069b155e9c0e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.737689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2d276-37a8-4ccf-9aa7-ebedfc6439e5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.754288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.755335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.755621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.755899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.756663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:15.756973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.758441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.776695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 298/599] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:15.776695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.781868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.782691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.783005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.783286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.791117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.792589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5d5cb57f-7b84-42b3-80fc-f14bb08978e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:15.804351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:15.805667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 302/600] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27810-978a-4966-9328-ad18bba973af None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.807807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27810-978a-4966-9328-ad18bba973af None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.808710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27810-978a-4966-9328-ad18bba973af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.828519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7ad27810-978a-4966-9328-ad18bba973af None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:15.829770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7ad27810-978a-4966-9328-ad18bba973af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:15.900457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78859967-3be6-4746-b21d-891e6b8e980c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:15.914873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 303/601] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:15.914873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27810-978a-4966-9328-ad18bba973af None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:15.931290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.939480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:15.940158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:15.940447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:15.940727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:15.943880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:15.952779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:15.954258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:15.971038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:15.971986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:15.972861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:15.972861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:15.972861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:15.972861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:15.972861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 299/602] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:15.973141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.038534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27810-978a-4966-9328-ad18bba973af None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.081340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.082206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34637c5-612b-43d2-8595-7d4a9f43324d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.082542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.083044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.083513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/a0c7dc51159c4fcd8ae520f82ef7ef61/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.103734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=a0c7dc51159c4fcd8ae520f82ef7ef61, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.105504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.122189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 304/603] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:15 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/a0c7dc51159c4fcd8ae520f82ef7ef61/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:16.123467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.135036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.136075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.136548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.137025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.138032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:16.138603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.140599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.157749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 300/604] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:16.159031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.225492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:16.246908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:16.295144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6dd6-7e52-4474-9c3b-e87898a29928 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.307408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b9296-f3f4-4fe0-b21e-11252b3c36c9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.353863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.354593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.354794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.355092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.355938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:16.356122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.357534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.360951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.361890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.362242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.362536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.370287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 301/605] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:16.370989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.375397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:16.381287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:16.383081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 305/606] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:16.384808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.495309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41636d5-7455-4718-b35b-27c63d26cfaf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.498581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c968bc9-cd2e-416e-9f07-022e790471ac None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.511269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:16.530862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:16.536575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.537570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.537777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.538194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.538464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.538764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/6146fcd207164df18b9c08e8fcc1adb6/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.538914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.539238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.547627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.549880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.557256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=6146fcd207164df18b9c08e8fcc1adb6, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.559020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:16.566800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:16.568464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:16.569692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:16.570850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 306/607] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:16.572008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.581006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 302/608] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:16 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/6146fcd207164df18b9c08e8fcc1adb6/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:16.588564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.765709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f18aa2-4a0e-4c2a-88e9-2a448434f4cd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.808438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f12915-2090-4843-8ee3-9d5bcdeab77c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:16.835517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.836425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.836553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.836989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.837645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:16.838166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.839725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.849698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 307/609] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:16.856895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.887581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:16.888466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:16.889234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:16.889528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:16.890590 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:16.890932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:16.893071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:16.910297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 303/610] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:16.916467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:16.925115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:16.935392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:17.017906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec771b6a-cd04-4b81-b1eb-4c31da52bc57 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.089287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c4ed0-fadb-4306-abd5-ffaea6cce964 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.089938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.090625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.091165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.091453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.106700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.108671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.125712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a4d97-0592-4827-a7f3-5491ab8bba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.126396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4d97-0592-4827-a7f3-5491ab8bba9b None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.126716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4d97-0592-4827-a7f3-5491ab8bba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a5bc5786-5aa3-4cce-bf54-4faabd97975e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:17.126983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:17.128803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:17.131187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:17.131187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:17.131187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:17.131187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:17.131677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4d97-0592-4827-a7f3-5491ab8bba9b None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/5fda0e3ee3624ac19de85bb3db30e2dd/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.131826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 308/611] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:17.131826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b82e-4f1e-4c5d-acc9-17f0611fc40b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.131826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b82e-4f1e-4c5d-acc9-17f0611fc40b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.131826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b82e-4f1e-4c5d-acc9-17f0611fc40b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.138870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 309/612] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:17.138870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b82e-4f1e-4c5d-acc9-17f0611fc40b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.145837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a4d97-0592-4827-a7f3-5491ab8bba9b None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=5fda0e3ee3624ac19de85bb3db30e2dd, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.147978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a4d97-0592-4827-a7f3-5491ab8bba9b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.178993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 304/613] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:16 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/5fda0e3ee3624ac19de85bb3db30e2dd/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:17.180585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.180905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.181237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.229939 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:17.231244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:17.315789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b82e-4f1e-4c5d-acc9-17f0611fc40b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.327243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:17.363996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:17.381589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 305/614] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:17.389085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.390020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.390473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.390929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.391779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:17.392231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.393747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.406760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.407625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 310/615] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:17.414017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.527204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220a9c5-0b04-4dad-9e8c-356d943065a0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.558493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153ef6a-f922-4347-968e-61eaef7d53b1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.567698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.568305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.568602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.568887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.590567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:17.595139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.595803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.596083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.596355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:17.603780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:17.604727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:17.606010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:17.606813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.606914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 306/616] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:17.606983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.611102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:17.620104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:17.622194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:17.623452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 311/617] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:17.632114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.670866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:17.702412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:17.741110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad721-8bb7-47ad-9a2f-6acdcb8a32e0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.790797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.790797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.790797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.790797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.791259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:17.791259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.791656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.797012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcef67-2a22-4c55-9621-6cfa15ea3e48 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.810287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 312/618] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:17.815825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.899732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.900387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.900677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.900947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.901704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:17.902014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.903447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.903581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:17.920332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50a4d5-b110-4e04-82b0-94b545a7695d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:17.929395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 307/619] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:17.939870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:17.956272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:17.957327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:17.957980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:17.958453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:17.969577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:17.971524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:17.987636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:17.988617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:17.989502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:17.989502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:17.989502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:17.989502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:17.991736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 313/620] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:17.997630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.104273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99ca07-fa05-491f-a537-e4e707f5c24a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.156539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b444a94-20b7-4bb1-90c2-938c0ab6e424 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.158475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.159420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.159902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.160370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.161294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:18.161794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.163415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.177164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:18.177818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:18.193735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.194427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.194715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.195076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.196816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 314/621] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:18.206889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.207085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.208351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:18.226867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:18.227864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:18.228727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:18.228727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:18.228727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:18.228727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:18.228727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 308/622] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:18.232928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.354000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dad92-7493-48bf-b060-b37901ab65f0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.381115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508f1cc-79c6-4afd-812d-b685b7ce74e8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.410941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.411605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.411897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.412284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.413001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:18.413368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.414872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.417088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.417747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.418033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.418307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.425435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.426929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.427868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 309/623] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:18.439067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:18.440965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:18.442028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:18.442558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.442645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 315/624] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:18.448755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.491735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:18.565089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:18.628486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e17c0-74d0-46ad-8f91-51350b8ca7a3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.647964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e866-0adb-4a62-a49e-fe831ad52193 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.693077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.693991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.694415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.695557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.706768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.710376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.712609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.713427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.713849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.714260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.715209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:18.715841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.718705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:18.739158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:18.740366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:18.741500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:18.750969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 316/626] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:18.752061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.752217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 310/626] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:00:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:18.757123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.865808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:18.888901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1169ca-385e-41e7-8ca5-9924d91db3e1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.894416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352977c5-ac6e-4b11-8d7a-13e2dd955aff None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:18.912425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:18.933101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.934162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.934587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.934899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3684a-bbf4-4dbd-9992-bdaa9a479ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:18.935602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3684a-bbf4-4dbd-9992-bdaa9a479ac5 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.935820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.935937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3684a-bbf4-4dbd-9992-bdaa9a479ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.936181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3684a-bbf4-4dbd-9992-bdaa9a479ac5 None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/b4423ec5dc2f465086495db19afde175/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.937242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:18.937791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.939650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1008b65-79e3-4a5c-905a-a3543cc60353 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.955874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3684a-bbf4-4dbd-9992-bdaa9a479ac5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=b4423ec5dc2f465086495db19afde175, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:18.957320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3684a-bbf4-4dbd-9992-bdaa9a479ac5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:18.963367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 317/627] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:00:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:18.964914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003e40-ebff-4216-9b7e-5b682ed94594 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.965348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003e40-ebff-4216-9b7e-5b682ed94594 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.965762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003e40-ebff-4216-9b7e-5b682ed94594 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:18.967383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 318/628] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:18.968415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003e40-ebff-4216-9b7e-5b682ed94594 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:18.982069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 311/629] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:18 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/b4423ec5dc2f465086495db19afde175/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:00:18.986868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:18.986868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:18.986868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.004410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:19.011027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:19.084929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 312/630] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:19.110441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:19.152722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003e40-ebff-4216-9b7e-5b682ed94594 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:19.191570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:19.221892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684039c-5509-4d90-afd7-3074a1e6ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:19.222934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684039c-5509-4d90-afd7-3074a1e6ac96 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.223324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684039c-5509-4d90-afd7-3074a1e6ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.223701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684039c-5509-4d90-afd7-3074a1e6ac96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.243085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff6392-bec1-4a94-9cfa-8fa1c1229cf5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:19.285676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bb9b7-cf3b-4d05-98d3-aacf8f709a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:19.286360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb9b7-cf3b-4d05-98d3-aacf8f709a41 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.286732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb9b7-cf3b-4d05-98d3-aacf8f709a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.287073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:19.289146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb9b7-cf3b-4d05-98d3-aacf8f709a41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.305635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bb9b7-cf3b-4d05-98d3-aacf8f709a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7f6bb9b7-cf3b-4d05-98d3-aacf8f709a41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:19.312743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:19.313693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:19.314506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 313/631] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:19.315976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f4b88-6929-4c7f-9046-e3ce5f6d416b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.316267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f4b88-6929-4c7f-9046-e3ce5f6d416b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.316543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f4b88-6929-4c7f-9046-e3ce5f6d416b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.318081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 314/632] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:19.320143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684039c-5509-4d90-afd7-3074a1e6ac96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:19.321677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684039c-5509-4d90-afd7-3074a1e6ac96 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:19.327307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.327586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.327867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.348194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:19.349399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:19.440624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 315/633] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:19.441889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:19.444026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 319/634] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 01:00:19.447714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48940f5-6a42-4d80-aa24-b15425e74867 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.447714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48940f5-6a42-4d80-aa24-b15425e74867 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.447714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48940f5-6a42-4d80-aa24-b15425e74867 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.450587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 320/635] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:19.450968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48940f5-6a42-4d80-aa24-b15425e74867 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:19.511699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:19.566052 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db7b2c-64c8-45e0-80ab-f3258fb86b61 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:19.611847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:19.612502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.612792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.613073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.613845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:19.614131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:19.615597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c2a30-f27e-4971-93df-bf5d0850eef8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:19.625599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 316/636] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:19.627009 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.627304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.627587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.646526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:19.647758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:19.670164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:19.702544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48940f5-6a42-4d80-aa24-b15425e74867 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:19.722061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 317/637] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:19.722949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:19.746203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:19.746777 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.747168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.747442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.758207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:19.759737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:19.775360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:19.776330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:19.777222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:19.777222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:19.777222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:19.777222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:19.777222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 321/638] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:19.777502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:19.784553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:19.844619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6f9cb-920e-48cf-9d09-62339b7df55a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:19.883320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bdd80-29c6-4123-a36d-6b37b8676210 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:19.886230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:19.886988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.887276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.887553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.900445 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:19.905616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:19.906797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:19.907756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 318/639] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:19.908931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:19.921396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:19.932889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:19.933579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:19.933940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:19.934224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:19.946370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:19.947913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:19.966826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:19.967821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:19.968691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:19.968691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:19.968691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:19.968691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:19.968691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 322/640] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:19.968976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.045609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1ea53-29eb-4c94-ba63-b0a079dfc689 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.054365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.084113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.084771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.085050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.085327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.086460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b18ce-84b5-4bc3-9539-2a9e5cb95b03 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.096575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:20.101560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:20.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:20.103542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 319/641] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:20.104264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.140301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.140936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.141227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.141492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.142657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:20.142657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.143874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.156649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 323/642] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:20.157396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.200552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.214959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318d533-f04c-4123-9b03-b797d1fe3d9c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.256185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.256872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.257123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.257403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.258306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:20.258493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.259960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.274359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 320/643] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:20.274359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.277477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5f43d-ca9a-40a2-8cc3-caf39bdbab71 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.315650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.317454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.318177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.318540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.318775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.320809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:20.320809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.321787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.341487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 324/644] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:20.342471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.402301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e373d5-5c38-444f-a7b6-a0d1f40ad099 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.442495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.443184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.443461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.443742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.447003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:20.447003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.447003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.466560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 321/645] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:20.466766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.470493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.480686 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359f4df-33c4-4d3c-9f3e-d9d8bee36d75 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.530422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.531765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.532285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.532720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.541734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.543746 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:20.557555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:20.558872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 325/646] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:20.560443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.606675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.621250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa605-18de-4409-8694-0543044efa22 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.660671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.661459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.661754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.662029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.674218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.675753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.687304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c78f0c-d7cb-4b53-8e67-80a2886b8e03 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:20.690810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:20.691678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:20.697242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:20.697242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:20.697242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:20.697242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:20.697242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 322/647] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:20.697242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.728748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.729423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.729711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.729991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.739429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.740884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:20.753928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:20.755595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 326/648] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:20.756598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.780571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.815493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f2e1-f730-4be2-a652-32c9b7574d24 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.852168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.853205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.853683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.854132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.866179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.867604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.869841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:20.881301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ccc93-79ad-4726-9329-35adcb5c878f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:20.884867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:20.885739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:20.886807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:20.886807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:20.886807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:20.886807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:20.886807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 323/649] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:20.893380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:20.931300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:20.932535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:20.933252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:20.933938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:20.934783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:20.935157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:20.936959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:20.955163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 327/650] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:20.955851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.011323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e5a51-6a35-4101-8777-7702eda7a591 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.042706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.059312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.059968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.060273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.060518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.061231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:21.061542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.063124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.067091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313392ad-e3e2-483f-958e-41be27064e40 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.080278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 324/651] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:21.080278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.108594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48906b50-6325-4c50-8871-549927471313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.109226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48906b50-6325-4c50-8871-549927471313 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.109499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48906b50-6325-4c50-8871-549927471313 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.109774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48906b50-6325-4c50-8871-549927471313 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.110578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48906b50-6325-4c50-8871-549927471313 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:21.110917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48906b50-6325-4c50-8871-549927471313 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.112351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48906b50-6325-4c50-8871-549927471313 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.133143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 328/652] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 01:00:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:21.133982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48906b50-6325-4c50-8871-549927471313 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.139326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.188326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47bd9-16c9-431c-899f-b2c0d5177437 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.232832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.233483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.233767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.234048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.234766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:21.235131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.236521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.250085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 325/653] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:21.251128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.262442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48906b50-6325-4c50-8871-549927471313 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.306067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.307060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.307342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.307580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.308395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8f2f77d4f4c44890ac1ab88ff66100bd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:21.308711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.310143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.313115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.316740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 329/654] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/role_assignments?user.id=8f2f77d4f4c44890ac1ab88ff66100bd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:21.319748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.380382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176294d-a0e5-42c4-b90e-e2f0243f261b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.396393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.421246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.422050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.422200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.422448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.431239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.432718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.444491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8d4d5-e563-4fad-acbb-6b1c4b9ab424 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-79cf9ac1-32e5-47ec-b5b5-e448a480dae7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:21.446905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:21.447934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 326/655] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79162-8d91-4fe9-aeb2-557a55634cf4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.448894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79162-8d91-4fe9-aeb2-557a55634cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.449592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79162-8d91-4fe9-aeb2-557a55634cf4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.451369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 327/656] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:21.451900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79162-8d91-4fe9-aeb2-557a55634cf4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.500826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.501575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.501924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.502276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.583773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79162-8d91-4fe9-aeb2-557a55634cf4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.584338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.588073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.590255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.620368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.621030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.621316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.621597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.630170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.631681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c79f12fd-ceb9-4cb8-b613-40febcfadaba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:21.644067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:21.645243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:21.646180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 328/657] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:21.647129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.647424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.647702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.666478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:21.667743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:21.696944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.712034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 330/658] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:21.713046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.746517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 329/659] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:21.747529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.834992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:21.838352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572f672-cb05-4216-aad6-f00434df30bb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.860927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597592eb-d0b9-4671-b691-0dff27794fb1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:21.877424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.878069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.878358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.878625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.879467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:21.880163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.881609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.905474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 331/660] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:21.905474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.908790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:21.909437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:21.909712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:21.909987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:21.910826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:21.911166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:21.912610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:21.937196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 330/661] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:21.938038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:21.963099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.019274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72251b-5fcf-4ae0-8f8b-5f03e4fd1bce None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.048551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6c94b-9150-4501-aba1-5082b6a4f4cb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.057297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.057974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.058256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.058533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.070496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.072112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:22.089795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:22.096757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:22.097675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:22.097675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:22.097675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:22.097675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:22.097675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 332/662] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:22.097675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.111598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.218211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.218829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.219163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.219482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.223056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2be6-b3d8-4f5f-a5bb-466119d95fec None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.231563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:22.236900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:22.237872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:22.238781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 331/663] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:22.239495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.270483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.271412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.271876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.272153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.336443 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.351228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.353205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:22.355326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5164-83bc-40e7-ba3c-7b200ca447bf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.367858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.395937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.396874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.397159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.397440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.471584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 333/664] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:22.472812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.481767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.483304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:22.575030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.589178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06051c2-215c-4615-8b1e-13b14115f003 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.598094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 332/665] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:22.598879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.601896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.635385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.636302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.637033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.637330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.638331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:22.638648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.640811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:22.656537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 334/666] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:22.657469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.731441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63dcc62-3a9d-48f0-a87e-bf881afeea2e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.791766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.792407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.792690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.792967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.793699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:22.794000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.796965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:22.797258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e539b-2c9b-4cde-a9c4-661f981885c6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.809317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 333/667] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:22.810318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.840185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.841170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.841458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.841743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.853984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.855484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:22.860316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:22.870582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:22.871643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:22.872692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:22.872692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:22.872692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:22.872692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:22.880056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 335/668] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:22.880056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:22.911008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:22.924833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421fb4a-f905-439b-ac79-2997e3d7c34d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:22.973414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:22.974086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:22.974368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:22.982933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:22.987402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:22.989560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:23.012609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:23.013779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:23.014865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:23.014865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:23.014865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:23.014865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:23.018670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 334/669] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:23.018670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.022875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28f654-f0d7-424d-9272-081d17019c01 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.059308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.060328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.060776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.061060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.068914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.070492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:23.083900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:23.084988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:23.085868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 336/670] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:23.086360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.143681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.152608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92b65d-191f-4c2b-9c75-6bbb2e561afb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.182949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.202759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc905a9-fa2c-4933-8d47-fb20397dea45 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.212244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.213684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.214585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.215227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.231444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.235673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.244824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.245481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.245767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.246040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.246780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:23.247120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.248668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:23.254244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:23.255816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:23.262942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:23.262942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:23.262942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:23.262942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:23.262942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 335/671] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:23.262942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.269763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 337/672] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:23.270705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.380319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a05a-3a3d-4a21-bb56-0ed44f14ea5c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.385061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9db6c-bade-4898-b978-5dc015af67e8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.422096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.422823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.423164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.423553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.423668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.423987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.424094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.424189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.424646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:23.424890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:23.424972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.425176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.426329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.426600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.439542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 336/673] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:23.440639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.441084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 338/674] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:23.442014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.449476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.462640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.556985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6528b65-71b7-47e8-a8d5-4fb437bdcaeb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.562702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be7593-cdd4-448c-aca7-0f1c2b7b7884 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.595709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.596452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.596895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.597092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.600432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.601034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.601317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.601593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.602333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:23.602637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.604071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9af80-600f-4bb9-878c-f109951e0709 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.604362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.605814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.616980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 337/675] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:23.617862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:23.619006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:23.619922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:23.620416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ad0f-743b-43d3-8fb7-6b5e8f062ef8 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.620416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ad0f-743b-43d3-8fb7-6b5e8f062ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.620416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38ad0f-743b-43d3-8fb7-6b5e8f062ef8 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.620725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 339/676] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:23.620725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.622650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 338/677] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:23.624288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38ad0f-743b-43d3-8fb7-6b5e8f062ef8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.689106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.694109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.728668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1f1b6-958a-43b4-922e-e5ecc1b5673e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.729756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38ad0f-743b-43d3-8fb7-6b5e8f062ef8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.763507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.764141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.764424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.764699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.764918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.765493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=a0c7dc51159c4fcd8ae520f82ef7ef61) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:23.765602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.765803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.765898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.766097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.767279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.773120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.773301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 340/678] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/role_assignments?user.id=a0c7dc51159c4fcd8ae520f82ef7ef61&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:23.773928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.774595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:23.787639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:23.788961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:23.789860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 339/679] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:23.790451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.900513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad1170-c799-4e54-bae6-3c7d2fd37261 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.930569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.932350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2934-517c-4d97-b15e-c5a505caa894 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:23.939497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.941085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.941085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.941085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.951587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.953057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.965221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b97c9253-a0c2-4ab6-8198-3514ec2724ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:23.966706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:23.968080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:23.969078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:23.971277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 341/680] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:23.972569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.973856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.973856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.977741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:23.978117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:23.978702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:23.978702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:23.979476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:23.979795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:23.981256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:23.995031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:23.995224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 340/681] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:23.995442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:23.995562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:24.083192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 342/682] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:24.084360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.110042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e900b-efeb-4c1c-a9d1-c2243986e2a1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.150055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.150787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.151039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.151313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.152077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:24.152391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.153833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.166484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 341/683] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:24.167454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.203557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d5c9a-298f-4fb7-8336-c16b04634c39 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.222038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:24.222231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:24.241474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.242110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.242391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.242663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.243529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:24.243840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.245319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.263369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 343/684] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:24.264217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.281039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a03d4-4919-42e2-9921-bf747acd58f8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.318528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.319232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.319516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.319797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.320607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=6146fcd207164df18b9c08e8fcc1adb6) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:24.320960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.322432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.338867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 342/685] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/role_assignments?user.id=6146fcd207164df18b9c08e8fcc1adb6&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:24.338867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.394390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acae5d4-4722-4ed4-952a-172c3a401ec2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.444863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.449624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.450004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.450345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.460561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fc6fb-3224-493c-839e-096ea4bb96c3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.466593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:24.477025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:24.478048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:24.478915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:24.479953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 344/686] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:24.480828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:24.481629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.507630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.508288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.508618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.508893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.509730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=5fda0e3ee3624ac19de85bb3db30e2dd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:24.510149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.511964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.518926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 343/687] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/role_assignments?user.id=5fda0e3ee3624ac19de85bb3db30e2dd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:24.519874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.535421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:24.632100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73771ab7-5a0f-4fda-ba48-59e13046a138 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.667368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274af67-b05b-4dfb-b17b-7d738e5050f9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.690308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.691026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.691325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.691602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.717810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.718826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.719370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.719855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.720856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:24.721491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.723188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.733022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 344/688] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:24.738827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.788601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.790910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:24.791640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.847289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:24.862047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786285c8-8cb4-4916-aa34-b0b5f69ca046 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:24.903628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:24.904292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:24.904604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:24.904861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:24.919460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:24.920950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:24.931032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 345/689] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:24.938336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:24.943222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:24.944114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:24.944988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:24.944988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:24.944988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:24.944988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:24.944988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 345/690] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:24.952427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:25.120030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:25.132628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad534-e116-4276-bd63-a839f68ce657 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.136161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f656e6c-1fcf-4a8d-8e17-0a57f46dcacd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.196091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.197566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.197566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.197566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.203151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:25.214997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.216008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.216491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.216962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.217882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:25.218418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:25.220023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:25.221302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:25.224478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:25.250378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 346/691] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:25.256864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:25.258868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:25.259757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:25.260736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:25.260736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:25.260736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:25.260736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:25.263158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 346/692] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:25.271870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:25.372779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857891-a2cf-4301-ade4-174fdd3748b0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.412440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.413102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.413385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.413661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.428193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:25.429672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:25.454553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:25.455630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:25.456577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 347/693] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:00:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:25.463881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:25.491755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:25.500437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152326a5-46a8-4996-b972-e3086b7570e8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.543690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.544395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.544684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.544960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.545685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:25.545992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:25.547420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:25.562795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 347/694] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:25.572048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:25.633784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930a010-d947-45f9-bb2b-9e939307ed81 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.686982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.688709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:25.689688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.690188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.690872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.692837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:25.694441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:25.697206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:25.739304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 348/695] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:00:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:25.741268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:25.840762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:25.879437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491247d1-b4a9-467f-922b-9d03d6bf8c19 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.884935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb5a70-671e-4574-9bce-22e5a909a1d8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:25.933876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c2c6-4e5c-457a-9b81-2a42cbe978c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.934515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c2c6-4e5c-457a-9b81-2a42cbe978c4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.934795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c2c6-4e5c-457a-9b81-2a42cbe978c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.935105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c2c6-4e5c-457a-9b81-2a42cbe978c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.955100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:25.955717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:25.956023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:25.956299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:25.972678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:25.973800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:25.986069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:25.987035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:25.987941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:25.992441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 348/696] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:25.993855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.059870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c2c6-4e5c-457a-9b81-2a42cbe978c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.061480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c2c6-4e5c-457a-9b81-2a42cbe978c4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.118214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621038e-26aa-4eff-a582-fe78a370dcb2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:26.128140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:26.144953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:26.162353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:26.163022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.163296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.163570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.164598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:26.164948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.166532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df49e-3361-4be0-9608-caa39d0226a2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.182924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 349/697] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:26.184523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973c6f4-8ddd-4389-8273-9ebe73d3233b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.185050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973c6f4-8ddd-4389-8273-9ebe73d3233b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.185581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973c6f4-8ddd-4389-8273-9ebe73d3233b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.191169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 350/698] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:26.192279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973c6f4-8ddd-4389-8273-9ebe73d3233b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.198972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 349/699] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:26.200327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.200617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.200892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.221067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:26.222354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:26.298881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 350/700] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:26.298881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.323043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973c6f4-8ddd-4389-8273-9ebe73d3233b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:26.374730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:26.375485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.375830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.376116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.377109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=b4423ec5dc2f465086495db19afde175) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:26.377424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.379576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.390932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 351/701] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:26 2022] GET /identity/v3/role_assignments?user.id=b4423ec5dc2f465086495db19afde175&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:26.392225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.443278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:26.452493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5db50-fda6-4211-a75a-ce212dcebb95 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:26.471719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:26.521144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:26.521825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.522106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.522389 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.540007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.541555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.559701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcafda-201e-43b3-ac92-a8f9e6e81f33 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:26.571544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:26.572492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:26.573354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:26.573354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:26.573354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:26.573354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:26.577523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 351/702] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:26.578968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.603812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:26.604884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.605432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.605907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.630668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.633721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:26.673155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:26.674109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:26.675059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:26.675059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:26.675059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:26.675059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:26.675059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 352/703] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:26.680857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.765447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1c70d-540c-4d2d-ab68-974f5a2e6756 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:26.807902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:26.815238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:26.815966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.816319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.816665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.817452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:26.817830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.820192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.832815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6c5f7-b361-4566-a2b0-f6b43ace2048 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:26.838198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 352/704] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:26.843527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.863608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:26.883642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:26.884396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.884784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.885160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.886085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:26.886789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:26.888172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfb3f7-78ef-49a5-9293-63734d0c193c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:26.907012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 353/705] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:26.908570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667a64e-65f6-45a9-84bc-757a97e02621 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:26.908865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667a64e-65f6-45a9-84bc-757a97e02621 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:26.909166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667a64e-65f6-45a9-84bc-757a97e02621 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:26.910882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 354/706] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:26.911830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667a64e-65f6-45a9-84bc-757a97e02621 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:26.973338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975e531-77df-49b8-95a7-f02b46c6a8b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.012644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.013307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.013592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.013866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.018987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667a64e-65f6-45a9-84bc-757a97e02621 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.021871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.023836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4d4c8f86-17af-4549-b53c-7cfb0e3123f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:27.037491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:27.038479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:27.039595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 353/707] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:27.040604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebde80-19ef-4d6f-b08b-14f9570641b9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.040883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebde80-19ef-4d6f-b08b-14f9570641b9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.041161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebde80-19ef-4d6f-b08b-14f9570641b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.056091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891b208-a99f-43b2-86b0-ca009b417eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.056747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891b208-a99f-43b2-86b0-ca009b417eff None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.057032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891b208-a99f-43b2-86b0-ca009b417eff None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.057310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891b208-a99f-43b2-86b0-ca009b417eff None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.058021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891b208-a99f-43b2-86b0-ca009b417eff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.059500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891b208-a99f-43b2-86b0-ca009b417eff None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.060858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-09ebde80-19ef-4d6f-b08b-14f9570641b9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:27.062246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-09ebde80-19ef-4d6f-b08b-14f9570641b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:27.077426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:27.092197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 355/708] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:00:26 2022] POST /identity/v3/users => generated 254 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:27.094291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7dc7a-0201-44bc-88c0-fbc5583c5849 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.095448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7dc7a-0201-44bc-88c0-fbc5583c5849 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.095781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7dc7a-0201-44bc-88c0-fbc5583c5849 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.102382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 356/709] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:27.103452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7dc7a-0201-44bc-88c0-fbc5583c5849 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.146264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:27.159060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 354/710] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:27.162651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.162984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.163266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.190455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:27.191749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:27.240536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7dc7a-0201-44bc-88c0-fbc5583c5849 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.279284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 355/711] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:27.280024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.304783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.304783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.305071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.305367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.306145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:27.306458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.308067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.321382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 357/712] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:27.321936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.393821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:27.409260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d7183-c901-4e37-bcaa-22116f434cb1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.451186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bde56-1314-44cd-9f17-89ab41873c5b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.457413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:27.469105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.474438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.475096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.476231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.477993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.480763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.490170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.490975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.491335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.491683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.492403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.493960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:27.495923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:27.496876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:27.497780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:27.500550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 356/713] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:27.502016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:27.510601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:27.511911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:27.512966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:27.519559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 358/714] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:27.519559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.626577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a83ae-7cb4-469e-b084-a4e4e3ee9fe6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.654735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7ee97-c180-425f-a0e3-e2cba57e50a6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.665455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.666239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.666669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.667419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.668135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8f2f77d4f4c44890ac1ab88ff66100bd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:27.668572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.670222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.682920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 357/715] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/role_assignments?user.id=8f2f77d4f4c44890ac1ab88ff66100bd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:27.684094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.686992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:27.699814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.700503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.700792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.701070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.701771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:27.702078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.704401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.713747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 359/716] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:27.718454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:27.768240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:27.866928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544ef46-b1ad-4f15-b06b-6cf0f0ecd4dd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.886129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd864d04-ea14-4bab-8cec-e8a6a56e9472 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:27.923324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.924384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.924875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.925386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.926284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:27.926797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.928532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.935650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 360/717] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:27.949329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:27.949997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:27.950282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:27.950560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:27.951855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] RBAC: Adding query filter params (name=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:27.952224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:27.953826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40b4e-b08a-47f3-92fb-d4cadc7ac23a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:27.968067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 358/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:27.973752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:28.062046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:28.137380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29801b0-0de4-47e6-86f0-3d6d7711ae9a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:28.174205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:28.175265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:28.175782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:28.176261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:28.177158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] RBAC: Adding query filter params (type=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:28.177667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:28.179265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74346a61-152b-4ca4-a47c-e9b1efa1aa69 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:28.184860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 361/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:28.973990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2e89-55e4-445f-a5e9-e09fc37f2b2b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:28.974533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2e89-55e4-445f-a5e9-e09fc37f2b2b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:28.974533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2e89-55e4-445f-a5e9-e09fc37f2b2b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:28.976169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 359/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:28.981908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:28.982750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:28.983262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:29.011868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:29.013462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:29.110277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 362/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:29.148685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2e89-55e4-445f-a5e9-e09fc37f2b2b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:29.284014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2e89-55e4-445f-a5e9-e09fc37f2b2b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:29.319209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:29.326188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:29.327131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:29.327423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:29.327703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:29.344286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:29.349325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:29.350263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:29.351148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 360/722] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:29.359992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:29.441040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:29.529128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224d87-270d-4637-bda5-95dcf015b7eb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:29.577044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:29.577609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:29.577897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:29.578170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:29.578938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:29.579249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:29.580631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08dfca-323c-4a13-a6ce-ac9a832c85b6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:29.595657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 363/723] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:29.602153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:29.631569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a5aa-194d-441b-9d17-6999cd638355 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:29.631919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a5aa-194d-441b-9d17-6999cd638355 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:29.632189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640a5aa-194d-441b-9d17-6999cd638355 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:29.633950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 364/724] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:29.639345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:29.639638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:29.639920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:29.665733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:29.667010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:29.747362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880df8ab-b1a6-40cd-96ad-0e7a89a57339 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:29.823302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:29.824040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:29.824326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:29.824600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:29.825627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 365/725] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:29.840868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:29.842264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:29.867784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:29.868841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:29.869741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:29.869741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:29.869741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:29.869741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:29.869741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 361/726] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:29.872353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:29.883535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:29.936720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:29.947233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.046989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83cfdf-b904-425d-bbca-2ef275c15ce0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.069202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63a345-4d3b-4510-b9cc-7ed02b099a9f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.091627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bb140-e268-4e67-ae6c-f86ae5ad775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.092668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bb140-e268-4e67-ae6c-f86ae5ad775d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.093240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bb140-e268-4e67-ae6c-f86ae5ad775d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.093709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bb140-e268-4e67-ae6c-f86ae5ad775d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.114899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bb140-e268-4e67-ae6c-f86ae5ad775d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8a3bb140-e268-4e67-ae6c-f86ae5ad775d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:30.121995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:30.122948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:30.123788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:30.124921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 366/727] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:30.125716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bf9f-ad69-433d-b3af-4316afe20b21 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.126008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bf9f-ad69-433d-b3af-4316afe20b21 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.126294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bf9f-ad69-433d-b3af-4316afe20b21 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.128618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 367/728] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:30.131676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bf9f-ad69-433d-b3af-4316afe20b21 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.140722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.141363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.141653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.141930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.142643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:30.143408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.144385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bb45e-7dfe-4147-87cc-ab2de7902f1f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.158610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 362/729] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:30.159973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.160266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.160547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.179190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:30.180242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:30.237421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.246100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bf9f-ad69-433d-b3af-4316afe20b21 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.263131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 363/730] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:30.264053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.283484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.284509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.284987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.285449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.286396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:30.286966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.288546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.298039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 368/731] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:30.304669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.306204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.393883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b63283-772b-4cd7-8be2-283f0ed67e00 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.431931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.432623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.433118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.433381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ecce2-728f-4671-9811-cca97eee45e5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.433553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.441466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.442971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:30.455750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:30.456753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 364/732] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:30.458075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.481882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.482660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.483052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.483552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.498697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:30.505331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:30.506291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:30.507166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 369/733] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:30.507890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.530415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.587092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.608386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696df2fb-a73b-497b-af61-ba1a264f839d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.621927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50e1bc-5704-4136-b3e6-b05369558687 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.651371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.652144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.652491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.652834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.662810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.663474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.663761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.664038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.664477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.664889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:30.665199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.666034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.666681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb027d8-bce5-4c45-95a9-8be512966fd0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:30.681494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:30.682515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:30.683449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:30.683449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:30.683449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:30.683449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:30.683449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 365/734] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:30.683449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.690822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 370/735] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:30.690822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b8918-bb7f-4ef0-94f4-85d15d5f9080 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.690822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b8918-bb7f-4ef0-94f4-85d15d5f9080 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.690822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b8918-bb7f-4ef0-94f4-85d15d5f9080 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.690822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 371/736] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:30.690822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742040c-7b2e-4631-8750-e41e1aa6b47e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.691374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742040c-7b2e-4631-8750-e41e1aa6b47e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.691374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742040c-7b2e-4631-8750-e41e1aa6b47e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.692613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 372/737] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:30.693326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742040c-7b2e-4631-8750-e41e1aa6b47e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.772971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.791650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a434b6a-67a4-42fd-803c-d8440015c0ff None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.797646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742040c-7b2e-4631-8750-e41e1aa6b47e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.827687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.828317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.828604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.828858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.829577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:30.829880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.831288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.832187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.832812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.833097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.833403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.833562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:30.834090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:30.834400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:30.835910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453e0fe-3fbc-4a75-a0a3-611534f1d947 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:30.840778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 366/738] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:30.841592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:30.852845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 373/739] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:30.852845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b69d2-de50-4929-860c-0995c6960b2a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.852845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b69d2-de50-4929-860c-0995c6960b2a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.852845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b69d2-de50-4929-860c-0995c6960b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.869943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3b8b69d2-de50-4929-860c-0995c6960b2a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:30.871194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b8b69d2-de50-4929-860c-0995c6960b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:30.940009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 374/740] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:30.941386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.941684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.941965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:30.944095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a934e6-10d9-4d0c-a4a1-7fc7b2c6b099 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:30.960140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:30.961446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:30.978400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:30.979048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:30.979331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:30.979601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.024389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.032165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 375/741] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 01:00:31.032996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.047226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.052680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.054153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.138375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c7b0-7f3e-44b4-98fd-abb92b4baef5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.154773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 367/742] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 01:00:31.155650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.179511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.180206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.180585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.180874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.191368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.192784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:31.207541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:31.208614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:31.209529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:31.209529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:31.209529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:31.209529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:31.209529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 376/743] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:31.209844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.257156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.261328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.261489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e6253-5286-4a70-bf49-bd3906365092 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.298028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.298777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.300492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.300492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.308086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.309540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.317913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92d8d7-5e5e-4c36-bcb8-3f2df9d6c860 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:31.322282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:31.323438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:31.324571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:31.327001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 368/744] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:31.328011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.355141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.355795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.356081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.356354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.357087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.358504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.377808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 377/745] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:00:31 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:31.378964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.455490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f70c72-6c3e-4214-9202-2b99a6f2dec6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.509611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.510628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.511158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.511647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.512982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.514553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.533939 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 369/746] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:00:31 2022] POST /identity/v3/services => generated 240 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:31.535398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.551231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08dd8-69ad-42e6-a6d5-ce37748aad3a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.564733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.566169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.619288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.619689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.619983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.620884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.641309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.643527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:31.664005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:31.665336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:31.666311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:31.666311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:31.666311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:31.666311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:31.669569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 378/747] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:31.671128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.705702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57991818-a69c-4ad7-aa50-0f242fb1f4bd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.746505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.747259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.747473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.747743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.748448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:31.748751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.750121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.765776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 370/748] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:31.766721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.805611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92bbc9-b6b9-42d5-9327-d3b266f649d9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.842515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.843146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.843476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.843702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.844512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:31.844816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.846419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.863272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.865310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 379/749] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:31.866380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.883428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:31.892573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cf995-a426-4b67-9aeb-ca4e7381f3cb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:31.932431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:31.933482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:31.934001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:31.934478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:31.935562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:31.936085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:31.937751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:31.951669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 371/750] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:31.952947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:31.982876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e79d05-99af-473a-a8f0-fe06bca94bc1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.019879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.020541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.021105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.021387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.029369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.030817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:32.043296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:32.044323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:32.045260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 380/751] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:32.045866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.073155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954a4bd-f99f-4d27-b2ed-e35f53fac14c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.111161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.112209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.112712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.113230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.116651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.133151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.160792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5da7ed-6a3c-4983-8341-65bcbad602ca None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.194637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.197151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.198055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.198769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.199081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.199358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.207436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.208604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:32.224250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:32.225240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:32.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:32.227092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 381/752] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:32.227993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.310968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 372/753] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:00:32.312267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.336846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d1a6c-c43d-479e-ba7b-ceba76304875 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.349335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.374656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.375388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.375576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.375855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.376662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:32.376982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.378399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.380603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.402613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 382/754] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:32.403571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.425169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38851f51-e56c-474b-a9d5-009dd73183a2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.464278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.465319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.465818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.466357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.467460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:32.467981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.469847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.484234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 373/755] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:32.485617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.510196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937d7da-0247-4fae-8108-816d2ab4f6ef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.544764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.545423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.545705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.545976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.556592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.558059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:32.574006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:32.574974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:32.575954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:32.575954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:32.575954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:32.575954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:32.579124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 383/756] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:32.580045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.595209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.599447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d8fc5-6b52-4796-94d6-43e1e5089067 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.631761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.643784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.644707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.645167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.645588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.658143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.659908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:32.676163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:32.677308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:32.678390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:32.678390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:32.678390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:32.678390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:32.680499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 374/757] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:32.681464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.700331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd002-ca7c-4e3f-92d2-ca2d32ad2146 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.735346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.735696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.735981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.736257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.737064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=b4423ec5dc2f465086495db19afde175) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:32.737373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.738820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.745243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 384/758] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/role_assignments?user.id=b4423ec5dc2f465086495db19afde175&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:32.746243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.807612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcec65-cf09-4dc4-8af0-71416ac6b1f6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.849641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.850685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.851228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.851707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.852703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:32.853220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.854923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.869767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 375/759] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:32.875793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:32.887017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.905843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369afc0-82ac-4cbd-baa8-1761a44fa57b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:32.920304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:32.976299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:32.977387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:32.977877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:32.978379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:32.979414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:32.979945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:32.981529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:32.995242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 385/760] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:33.000927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:33.035469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55618-db0d-4826-82b7-d3a250f0f1d4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:33.096014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:33.097097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.097599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.098087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.106242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:33.108077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:33.121142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:33.122093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:33.123026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:33.125511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 376/761] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:33.131811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:33.203565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:33.242679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cd61d-6718-4fef-b71c-5516c3ff5dfa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:33.249086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef8f3-28b2-48a0-b674-43bedc0913a8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:33.281146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:33.282153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.282639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.283182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.284162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:33.284701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:33.286569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:33.300991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 377/762] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:33.307457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:33.339651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:33.340334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.340630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.340925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.356068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:33.357627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:33.381572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:33.382535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:33.383857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:33.386415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 386/763] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:33.391832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:33.419126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84ecdc-cb2a-499c-a297-7095999e42ec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:33.456254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:33.457254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.457765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.458237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.459301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:33.459866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:33.461605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:33.463827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:33.475271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 378/764] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:33 2022] GET /identity/v3/role_assignments?user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:33.496144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:33.637074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcc523-7832-455b-a4b1-5f68d97c0d70 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:33.672572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:33.673221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.673511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.673788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.674652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:33.674998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:33.676449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bd82f-ab35-45ca-a5fa-0d9ccb577596 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:33.690082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 387/765] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:33.695707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:33.775120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08370d-48f0-4c50-9e61-ff9e175aec16 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.775444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08370d-48f0-4c50-9e61-ff9e175aec16 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.775683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08370d-48f0-4c50-9e61-ff9e175aec16 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.777309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 388/766] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:33.784292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea427-0ee6-4d17-8be6-e36f8bce46f7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.784621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea427-0ee6-4d17-8be6-e36f8bce46f7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.784992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea427-0ee6-4d17-8be6-e36f8bce46f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.808789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0fbea427-0ee6-4d17-8be6-e36f8bce46f7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:33.810122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0fbea427-0ee6-4d17-8be6-e36f8bce46f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:33.832697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c5b70-385a-4313-869a-8f7adf86a93a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:33.900139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00521a-13d6-4d8a-8ea1-8d972164bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:33.900890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00521a-13d6-4d8a-8ea1-8d972164bbae None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.901181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00521a-13d6-4d8a-8ea1-8d972164bbae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.901463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00521a-13d6-4d8a-8ea1-8d972164bbae None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8f2f77d4f4c44890ac1ab88ff66100bd/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.904641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00521a-13d6-4d8a-8ea1-8d972164bbae None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=8f2f77d4f4c44890ac1ab88ff66100bd, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:33.906121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00521a-13d6-4d8a-8ea1-8d972164bbae None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:33.930619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 389/767] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:33.931116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 379/768] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:33 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8f2f77d4f4c44890ac1ab88ff66100bd/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:00:33.940573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f5858-a989-48bb-9aaa-17dad983fe38 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.940929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f5858-a989-48bb-9aaa-17dad983fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.941142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f5858-a989-48bb-9aaa-17dad983fe38 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.942966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 390/769] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:33.949414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:33.949760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:33.950049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:33.975473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:33.976763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:33.997736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f5858-a989-48bb-9aaa-17dad983fe38 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.035898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:34.059288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:34.107212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 380/770] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:34.132611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.182924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f5858-a989-48bb-9aaa-17dad983fe38 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.227396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:34.228159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:34.228449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:34.228728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:34.229698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:34.231444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:34.243348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:34.244348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:34.245226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 391/771] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 01:00:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:34.250707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.269669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3392f85-81ee-4970-9b1d-9620c5c551c8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.339863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:34.341578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:34.341862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:34.342137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] PATH_INFO: `/v3/services/network` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:34.343016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:34.344412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:34.360925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:34.361974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:34.362797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 381/772] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:34.371188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.415217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:34.441874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:34.455358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238aec3-95a1-43dc-a4e0-92a4b5fbd0d1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.530555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf6e6d-798a-4a52-b7ed-1d7259556f5e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.532215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:34.533106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:34.533612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:34.534089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:34.535042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:34.535578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:34.537166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:34.547012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 392/773] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:34.548898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.568213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:34.568850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:34.569133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:34.569409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:34.570098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] RBAC: Adding query filter params (name=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:34.570456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:34.572018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:34.577250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 382/774] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:34.582935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.682290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a74a-5e1f-4ad2-9b12-269f4ce6461e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.740099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:34.740724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:34.741009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:34.741283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:34.741983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:34.742287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:34.743727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:34.756460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 393/775] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:34.761326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.785021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1186f6-ee49-4b8e-8742-69302bc4f60d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.805265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:34.811194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:34.835635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:34.836265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:34.836613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:34.836885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:34.837574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] RBAC: Adding query filter params (type=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:34.837877 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:34.839276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:34.846259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 383/776] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:34.852072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:34.944400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd7435-ae19-4f03-bd74-694ec4f85286 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:34.968622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367aa0f-11ae-4a63-b0e9-f3c9e4f79366 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:35.010775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.011466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.011745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.012017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.012818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] RBAC: Adding query filter params (interface=public, service_id=1f02988040d74fce857361ca5ce17b38, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:35.013130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:35.014578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:35.023705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 384/777] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/endpoints?service_id=1f02988040d74fce857361ca5ce17b38&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:35.029227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:35.030683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.031470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.031764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.032044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.032833 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] RBAC: Adding query filter params (interface=public, service_id=433221dafbbe42b393413a058d7aea4a, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:35.033148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:35.034552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:35.054143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 394/778] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:00:34 2022] GET /identity/v3/endpoints?service_id=433221dafbbe42b393413a058d7aea4a&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:35.059913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:35.072290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:35.155390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1e58e-3b8c-4513-becb-eead41c1a71b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:35.183273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:35.196447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa412e-a0a5-4139-bd21-9b47a21e20c8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:35.218871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbab0b-8af2-4288-a481-a4d7486bb538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.218871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbab0b-8af2-4288-a481-a4d7486bb538 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.218871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbab0b-8af2-4288-a481-a4d7486bb538 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.218871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbab0b-8af2-4288-a481-a4d7486bb538 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.219371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbab0b-8af2-4288-a481-a4d7486bb538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:35.219371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbab0b-8af2-4288-a481-a4d7486bb538 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:35.227001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 385/779] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:00:35 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:35.228416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51678fdb-f6d3-4940-8077-1bff7caefbfe None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.228762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51678fdb-f6d3-4940-8077-1bff7caefbfe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.229053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51678fdb-f6d3-4940-8077-1bff7caefbfe None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.236759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 386/780] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:35.239375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.239663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.242865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.256040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.262972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.262972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.262972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.262972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:35.262972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:35.275596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 395/781] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:00:35 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:35.278725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:35.280051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:35.371033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 387/782] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:35.390258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:35.532183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc709-17ab-49a3-8292-9db167b58043 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:35.571152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.571981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.572382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.572663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.582039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:35.587661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:35.592960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:35.593897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:35.595225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 396/783] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:35.602177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:35.638658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:35.733206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be7d53-41d4-46cd-86ff-659ee5b7e8a3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:35.774655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.775335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.775622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.775905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.776652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:35.776961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:35.778365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:35.791304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 388/784] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:35.797179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:35.924921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e0d0-b3c1-45cd-8bf2-72515978efdb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:35.969153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:35.969820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:35.970153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:35.970437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:35.987049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:35.988522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a56989c3-ef83-4e82-9b42-8f4213522834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:36.003545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:36.006057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:36.008149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:36.008149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:36.008149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:36.008149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:36.008149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 397/785] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:36.014805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.146115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:36.159354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44b193-f6c2-4225-9fee-d75a57746ee4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.202944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e014e-8341-4ae5-8996-155e247e8d90 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.203285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e014e-8341-4ae5-8996-155e247e8d90 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.203488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e014e-8341-4ae5-8996-155e247e8d90 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.205056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 398/786] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:36.209571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:36.210258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.210583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.214997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.214997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:36.214997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:36.215460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5a966-567f-4656-8fc0-4a20a66b0232 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.215460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5a966-567f-4656-8fc0-4a20a66b0232 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.215460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5a966-567f-4656-8fc0-4a20a66b0232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.215896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:36.234828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 389/787] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:36.235807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.247960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2ab5a966-567f-4656-8fc0-4a20a66b0232 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:36.249548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2ab5a966-567f-4656-8fc0-4a20a66b0232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:36.344708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:36.374207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 399/788] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:36.381134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe3f83-e220-4671-af4a-802b84bdb0f3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.391215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534613e-4012-468e-a33e-3d2febb1088d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.391507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534613e-4012-468e-a33e-3d2febb1088d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.391843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534613e-4012-468e-a33e-3d2febb1088d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.393341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 400/789] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:36.399340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.399627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.399906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.420868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:36.422278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:36.433863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:36.434858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.435148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.435705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.445430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:36.449075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:36.462972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:36.463953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 390/790] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:36.465344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.484651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:36.517154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 401/791] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:36.519114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.603276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293be8e-39a3-4e50-a15e-88599d283ebe None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.632905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16c203-6ec1-427d-9b8a-10513a93a540 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.639534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:36.640221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.640515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.640776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.648332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:36.653710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:36.659017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:36.659927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:36.661130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 391/792] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:36.665946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.681759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:36.682913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.683420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.683886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.684960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:36.685512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:36.688032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:36.714376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 402/793] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:36.715715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.738872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:36.772358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1a760-27d9-47f0-af21-b64e14ad757b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.809767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:36.810438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.810774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.811074 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.822247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:36.827832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:36.828755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 392/794] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:36.829999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.841914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f451a-569e-4718-9261-f56e96ace667 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.882741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:36.883943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:36.884422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:36.884899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:36.885890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:36.886456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:36.888287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:36.898598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 403/795] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:36.900581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:36.943028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:36.959306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20739e-cdd4-470e-ae0b-b6b77b9bbb5f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:36.997723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.002553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.003372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.003988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.004274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.017920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.019512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.032325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc6bb0-5f48-44b5-8082-f9c75607ff7e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:37.034550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:37.035567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:37.036436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:37.036436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:37.036436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:37.036436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:37.040884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 393/796] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:37.042119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.077267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.078361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.079214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.079751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.080714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:37.081288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.083177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.093640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 404/797] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:37.094948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.174252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add1a9-e73c-4891-bf91-6512aff9a0bc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.204592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.214352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.215042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.215327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.215664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.218024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba1acd-09df-42ca-b6ee-9aa75e8bcace None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.227325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.228819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:37.244456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:37.245990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:37.247711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:37.247711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:37.247711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:37.247711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:37.250585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 394/798] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:37.251536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.266447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.267545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.268024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.268584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.269636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:37.270139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.272269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.287284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 405/799] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:37.288648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.295810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.389666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e705d9-37e6-4362-8b21-12986b57ac32 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.433958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.434653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.435034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.435379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.457411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e819404-d4d5-4f21-820d-36fd0baf0984 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.458168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.460282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:37.487879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:37.488807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:37.489661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:37.489661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:37.489661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:37.489661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:37.492557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 395/800] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:37.493639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.509902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.510733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.511594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.511594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.512315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:37.512687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.514392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.532054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 406/801] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:37.533127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.549700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.616461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.630767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540006cc-f82a-4901-a626-7b34f2d9e55f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.648185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fe6c-b2fd-4f6b-8fa9-5ddf1f524821 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.674352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.675159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.675444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.675722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.683513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.685008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.690666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.691255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.691534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.691812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.692559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:37.692835 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.694222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2661f7-a9ef-481e-ad53-f3e4992b9dfa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:37.698579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:37.699527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:37.700442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 396/802] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:37.701498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.715265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 407/803] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:37.717838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652808ef-cf0c-4967-b448-517dd8e837b3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.718454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652808ef-cf0c-4967-b448-517dd8e837b3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.718702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652808ef-cf0c-4967-b448-517dd8e837b3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.720524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 408/804] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:37.721026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652808ef-cf0c-4967-b448-517dd8e837b3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.805895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.815680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b03c99-4d30-4235-b9f8-6ace346401ba None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.840207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652808ef-cf0c-4967-b448-517dd8e837b3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:37.850174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.850857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.851155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.851442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.860027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.861490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:37.873587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:37.874617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:37.875595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 397/805] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:37.876173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:37.876264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:37.876334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.876428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.876854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.877447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:37.877766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:37.879231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac167d33-0104-4fea-a304-13a8006eda33 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:37.894497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 409/806] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:37.895890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ec1d5-4ac3-42e3-8135-b4bc310aa45f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.896189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ec1d5-4ac3-42e3-8135-b4bc310aa45f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.896481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ec1d5-4ac3-42e3-8135-b4bc310aa45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.903539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:37.917345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9c3ec1d5-4ac3-42e3-8135-b4bc310aa45f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:37.918561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9c3ec1d5-4ac3-42e3-8135-b4bc310aa45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:37.987340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 410/807] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:37.988567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac3719-d6c9-40bf-b5a2-a76722add6b4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:37.988862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac3719-d6c9-40bf-b5a2-a76722add6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:37.989187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac3719-d6c9-40bf-b5a2-a76722add6b4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:37.990589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 411/808] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:37.991146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac3719-d6c9-40bf-b5a2-a76722add6b4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.008751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e53a24-eb7d-43cc-b1bd-321e2d9f6237 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.054587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.055352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.055633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.055906 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.063941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.065428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.067941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:38.078183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:38.079279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:38.080199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 398/809] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:38.080772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.117014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac3719-d6c9-40bf-b5a2-a76722add6b4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.155096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.155794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.156080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.156355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.157181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:38.157500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.159008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae4ee-f893-4822-a3b9-56bf88da084c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.161031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.179232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 412/810] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 01:00:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:38.180914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.181209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.181503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.199968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1cf64-6d7c-4db2-99c1-8081feb799c2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.202087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:38.203382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:38.239022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.240028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.240318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.240601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/b4423ec5dc2f465086495db19afde175/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.242669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=b4423ec5dc2f465086495db19afde175, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.244186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.259907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 399/811] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:38 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/b4423ec5dc2f465086495db19afde175/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:38.260465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.290870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 413/812] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:38.290870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.363284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.401653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5685fba-3b7f-4115-b762-c506f29150f4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.427445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05531a-716c-4c88-a592-6fa072c3a1eb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.444370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.456520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.457399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.457700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.458433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] PATH_INFO: `/v3/services/network` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.459534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.462415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.476591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.477955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.478464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.479045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.480494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:38.480777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:38.482499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 01:00:38.484205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:38.485659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.485809 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 400/813] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:38.485809 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.486036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.508994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 414/814] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:38.509562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.613023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852ec2c-cd58-4faf-ad62-693e2676f974 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.618318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba04387-2ad6-4c6e-8266-ed260e8468eb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.631583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.655456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.656103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.656406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.656669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.660905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.661757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.662183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.662622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.663446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.665004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.667457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.668909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:38.674814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:38.675989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 01:00:38.677036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:38.684501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 415/815] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:38.684800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:38.685820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:38.686714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:38.686714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:38.686714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:38.686714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:38.686714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 401/816] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:38.687072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.687991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.742946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.800131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfae53-d10c-4dfa-b76d-ae644b2f1ab6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.800319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e30717-8f84-4ef4-8d46-86e12c147157 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:38.836324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.837015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.837300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.837503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:38.837617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.838140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:38.838367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:38.838642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:38.839414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] RBAC: Adding query filter params (name=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:38.839722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.841099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.850815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 416/817] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:38.850815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:38.891701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.911865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:38.913344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:38.962592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:38.964811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6c627-99b5-4402-8920-f38de10ea438 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.003096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.003746 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.004027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.004309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.005404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:39.005717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.007107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.011962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 417/818] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:39.012782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.035745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 402/819] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:00:39.037027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.123377 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.126825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e2fd8-9e8d-41ef-96f5-ab7bd45bd023 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.147402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a37d9-7cc3-472f-a45a-0a53e84c4a02 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.165036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.165683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.165974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.166249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.166990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:39.167302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.168677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.181151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 418/820] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:39.182057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.188335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.188989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.189276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.189601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.190296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] RBAC: Adding query filter params (type=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:39.190609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.192029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.202559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 403/821] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:39.203568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.214756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.298945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9cc2b-443c-4da6-b24e-8502d1531795 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.313633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e982-b0c7-4927-8475-ae8b87383768 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.335380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.336032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.336442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.336716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.337411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:39.337722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.339114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.344727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 419/822] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:39.345451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.354221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.355853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.356152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.356434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.373003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.374321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.374529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:39.392599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:39.393585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:39.394461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:39.394461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:39.394461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:39.394461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:39.394461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 404/823] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:39.394731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.460287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283709c7-e42e-4be5-abb4-e2ce38363fd9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.460894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.505628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.506580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.506580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.506864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.512574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dcc0a-61f2-453b-8e0e-e28b0c61289f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.514099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.515772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:39.527498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:39.528465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 420/824] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:39.529375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.556221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.556996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.557372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.557711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.558492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.559920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.594871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 405/825] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:00:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:39.594871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.640317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.661320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14b6b-be66-4cea-9f9b-402fcb0d9ea9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.722124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.722915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.723363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.723606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.724424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.726090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.733829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bb3d4-0e63-4174-954e-7e82e99f4091 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.753275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.759375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 421/826] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:00:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:39.761076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.785935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.787661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.788279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.788828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.789824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:39.790470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.792524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.811424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 406/827] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:39.817122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:39.910787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dff2-3212-4a46-babc-80608b676f36 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.959554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb9d8e-2cf5-42f4-a8c9-4d5aaa60ecee None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:39.961684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:39.962546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:39.964072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:39.964555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:39.965492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:39.965954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:39.967602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:39.971546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:39.987793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 422/828] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:39.994089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:40.016179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:40.016838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.017225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.017507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.025879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:40.027407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:40.039857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:40.040849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:40.041716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:40.042151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 407/829] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:40.047907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:40.088462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:40.115457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd5552-f896-4b88-a1fd-9359e271f5f5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:40.152396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:40.153097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.153383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.153655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8d5e9ee4b7f94c08bfb5eeb391bfa700/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.170752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=8d5e9ee4b7f94c08bfb5eeb391bfa700, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:40.172570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:40.173630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868abb-0c08-43ac-acde-ac37af2a2b32 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:40.191433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 423/830] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:39 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8d5e9ee4b7f94c08bfb5eeb391bfa700/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:40.212462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:40.213176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.213504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.213782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.214635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:40.215070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:40.216641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874ab3e-64e1-40d4-a56b-612aae26abc1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:40.233480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 408/831] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:40.242865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:40.303689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:40.314328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761778a-98dc-4518-afec-1d20931ba40c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.315075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761778a-98dc-4518-afec-1d20931ba40c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.315564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761778a-98dc-4518-afec-1d20931ba40c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.317533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 409/832] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:40.324227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.325015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.325518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.348711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:40.350717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:40.370313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64951b9-72ea-4b22-b981-c9b737d3eb64 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:40.417479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:40.418151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.418414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.418898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.419549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8f2f77d4f4c44890ac1ab88ff66100bd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:40.419864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:40.421358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:40.428091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 424/833] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:40 2022] GET /identity/v3/role_assignments?user.id=8f2f77d4f4c44890ac1ab88ff66100bd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:40.441609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 410/834] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:40.465194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:40.599545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfde7e-7c42-47cc-9ef1-cd193d54132d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:40.611747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:40.644687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:40.645431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.645718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.645993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.658306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:40.663997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:40.665120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:40.665980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 425/835] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:40.671636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:40.699024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:40.790258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586f047-fec7-47f4-aed6-4e665870814e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:40.831949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:40.832612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:40.832894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:40.833174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:40.833942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:40.834242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:40.835689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:40.854445 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 411/836] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:40.860655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:40.995419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cf824-1370-449f-a6ad-a4dc3d7cf6a6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:41.033731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:41.034402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.034684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.034995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.047973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:41.049547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:41.065998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:41.067058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:41.068228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:41.068228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:41.068228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:41.068228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:41.068228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 426/837] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:41.076037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:41.173193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:41.191023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fb71b-1b1d-4d0f-9550-70ec122a4f3a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:41.231013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:41.232044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.232547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.233033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.233972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:41.234499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:41.236184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:41.253905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 412/838] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:41.256081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:41.285596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:41.366251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21362b2-ec65-46d6-9d87-13a59295cdd6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:41.405726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:41.406395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.406682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.406993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.415121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:41.416601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:41.429729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:41.430825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 01:00:41.431805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:41.434950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 427/839] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:00:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:41.437546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:41.559525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d449f92-f2e1-46aa-a1cf-723fb2bf1683 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:41.595940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:41.596580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.596867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.597144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.597956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:41.598267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:41.599730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:41.622673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 413/840] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jul 28 01:00:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:41.625092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:41.717040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:41.729947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6292f6-6699-427e-b613-ce3fc027d71a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:41.768488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143c15f-f249-49f0-b278-94d4f72e7fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:41.769137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143c15f-f249-49f0-b278-94d4f72e7fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.769424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143c15f-f249-49f0-b278-94d4f72e7fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.769705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143c15f-f249-49f0-b278-94d4f72e7fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.840639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143c15f-f249-49f0-b278-94d4f72e7fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:41.842029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143c15f-f249-49f0-b278-94d4f72e7fa9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:41.935207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:41.953241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 428/841] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:00:41.954685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:41.957736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:41.966739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f200b6a-d3de-4dc0-ae88-3e845a8d49d0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.967101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f200b6a-d3de-4dc0-ae88-3e845a8d49d0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.967306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f200b6a-d3de-4dc0-ae88-3e845a8d49d0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.969007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 429/842] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:41.974029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:41.974309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:41.974583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:41.992736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:41.993930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:42.071109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 430/843] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:42.082188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35fd6-4ccd-41d2-8971-e3e75fbc35e6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.089147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.128317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.128894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.129456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.144716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.146711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:42.166744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:42.167728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:42.168617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:42.168617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:42.168617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:42.168617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:42.177424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 414/844] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:42.179425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.205259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab3191-110b-49f8-8667-d0ed1dc590e4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.220949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:42.245342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.246011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.246283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.246561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.247245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.248612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.258277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:42.258626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:42.259681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:42.260912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:42.264016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 431/845] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:42.266563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.318815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7128a-fd78-4002-b753-1fb53c3035da None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.355352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.355988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.356273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.356547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.357253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:42.357561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.358998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372741c5-359a-4154-8dc1-de04acc1520a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.372953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 415/846] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:42.374095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5b5b-8aab-4a72-931b-902465c2771e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.374345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5b5b-8aab-4a72-931b-902465c2771e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.374627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5b5b-8aab-4a72-931b-902465c2771e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.379072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 416/847] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:42.380545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5b5b-8aab-4a72-931b-902465c2771e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.389095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33acff4-cd15-4265-9a0e-9005f113ca4e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.425817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.426456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.426740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.427043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.427739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:42.428067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.429418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f106893-81ce-466a-bfad-ad82d315c86f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.438894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 432/848] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:42.438894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.438894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.438894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.454514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:42.455757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:42.493722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:42.502783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5b5b-8aab-4a72-931b-902465c2771e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.518485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:42.532556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 433/849] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:42.533361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.550698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.552012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.552516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.553262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.571537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.573297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:42.586713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:42.587764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:42.588769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 417/850] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:42.589397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.658508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fdc2f-8556-4089-8e4c-f0044918bcde None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.695103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.696233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.696520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.696947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.697655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:42.697961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.699381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.704426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 434/851] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:42.705095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.707579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22118af6-52c6-48bf-a5c2-537e1899ee40 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.751126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:42.751560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75041c14-ee02-499b-9f36-30ca91b0e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.753278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75041c14-ee02-499b-9f36-30ca91b0e496 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.753629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75041c14-ee02-499b-9f36-30ca91b0e496 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.753998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75041c14-ee02-499b-9f36-30ca91b0e496 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.767547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75041c14-ee02-499b-9f36-30ca91b0e496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-75041c14-ee02-499b-9f36-30ca91b0e496 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:42.773230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:42.774336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:42.775219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 418/852] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:42.776003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58172a70-5144-446e-b466-55429b825d21 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.777362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58172a70-5144-446e-b466-55429b825d21 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.777362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58172a70-5144-446e-b466-55429b825d21 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.783293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 419/853] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:42.783293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58172a70-5144-446e-b466-55429b825d21 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:42.823284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:42.839620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced4224-01ef-417c-a04f-f8f09bc089c8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.883938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.884590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.884881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.885195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.885975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:42.886292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.887827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425402e-d671-4e22-9fec-575c12b9d2aa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.901032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 435/854] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:42.902425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253607e-4058-427f-9155-83c6395cf8da None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.902732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253607e-4058-427f-9155-83c6395cf8da None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.903066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253607e-4058-427f-9155-83c6395cf8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.920048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58172a70-5144-446e-b466-55429b825d21 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:42.923493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9253607e-4058-427f-9155-83c6395cf8da None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:42.924723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9253607e-4058-427f-9155-83c6395cf8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:42.960343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:42.961182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:42.961334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:42.961557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:42.962304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:42.962697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:42.964057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:42.980247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 420/855] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:42.980247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.017677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 436/856] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:43.019036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253607e-4058-427f-9155-83c6395cf8da None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.051382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:43.113433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:43.122381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47b859-3a63-418e-8c77-a389b52d38ee None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.137650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253607e-4058-427f-9155-83c6395cf8da None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.171298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.171842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.172124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.172406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.173212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=b4423ec5dc2f465086495db19afde175) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:43.173529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.175007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.181787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 421/857] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:42 2022] GET /identity/v3/role_assignments?user.id=b4423ec5dc2f465086495db19afde175&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:43.182490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.198723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.199828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.200346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.200811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.216949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.219578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:43.242146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:43.243283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:43.244166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:43.244166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:43.244166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:43.244166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:43.248571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 437/858] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:43.250341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.328666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daff43-2ecb-485b-ac6f-7a371a99bcf9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.364634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:43.388414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.389668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.390443 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.391843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.392776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.394403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.402868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b80d-6fd0-4144-9838-1c634a2b259e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:43.420506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:43.421745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:43.422674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:43.425414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 422/859] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:43.431387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.455154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.455915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.456350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.456674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.457464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:43.457856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.459591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.468760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:43.476842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 438/860] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:43.483057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.574638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2cf43-84a2-4cf1-9f14-fff6ef713a56 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.615717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.616398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.616687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.617114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.617820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:43.618132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.619729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.622548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98e7ef-e6dc-45b3-8ef9-7ff49e630d72 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.625491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 423/861] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:43.632048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.662255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.663681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.664117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.664630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.672773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.674543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.677732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:43.698910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:43.700053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:43.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 439/862] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:43.704757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:43.731924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:43.750960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221011e6-2683-430e-83e5-7799260d2ccd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.793453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.794762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.795340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.795862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.796884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:43.797427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.799315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.806534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 424/863] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:43.864546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24ac0f-4c0b-43c5-ae07-3c577a40a16d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:43.908322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:43.909389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:43.909900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:43.910514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:43.911933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:43.912675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:43.915382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:43.949804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 440/864] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:43.951669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:44.064607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d311-a5af-45fc-a2cc-734638baa509 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:44.100586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:44.101224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:44.101512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:44.101834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:44.124726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:44.174553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:44.176072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:44.284915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 425/865] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:00:44.290985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:44.352925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:44.399871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cb125-7599-4c3d-814c-26f47cf8d9a4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:44.437856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:44.438545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:44.438853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:44.439143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:44.456795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:44.458288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:44.474066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:44.475066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:44.475963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:44.475963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:44.475963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:44.475963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:44.480083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 441/866] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:44.483987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:44.540857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:44.594410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411854-d39f-47a4-8faf-a708255a0d09 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:44.633678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:44.634313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:44.634597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:44.634899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:44.635605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:44.635910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:44.637278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edd20d-f6bc-4bdf-9e5a-ca127d228b54 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:44.649624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 426/867] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:44.655563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:44.758212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27726720-6439-4fb9-b059-33547dfff5db None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:44.775495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f959c-5004-4540-89ae-144eb956a4e1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:44.775984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f959c-5004-4540-89ae-144eb956a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:44.776100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:44.776182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f959c-5004-4540-89ae-144eb956a4e1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:44.781737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 427/868] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:44.783094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:44.783384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:44.783665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:44.802491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:44.802683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:44.803661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:44.803852 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:44.804350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:44.804818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:44.814646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:44.817122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:44.834356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:44.835367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:44.836328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:44.844332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 442/869] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:44.845973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:44.889755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 428/870] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:44.912028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:44.974988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:44.984702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c88e3-00b9-4470-a0cc-116779ef23b5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:45.026934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:45.027620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.028005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.028282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.029060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:45.029370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:45.031018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:45.045394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 443/871] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:45.051645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:45.058244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3788a-6193-4972-9abf-616f6d98b7de None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:45.082891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:45.107171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841396b-84c2-4222-8c44-703c401b5d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:45.107868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841396b-84c2-4222-8c44-703c401b5d40 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.108103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841396b-84c2-4222-8c44-703c401b5d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.108386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841396b-84c2-4222-8c44-703c401b5d40 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.109396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841396b-84c2-4222-8c44-703c401b5d40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:45.111068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841396b-84c2-4222-8c44-703c401b5d40 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:45.131790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 429/872] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:00:44 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:00:45.179104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbaac3-f8e8-4b46-9c8c-034b918d15e7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:45.227628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:45.228283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.228610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.228891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.229769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:45.230090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:45.231680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425c3c7-e441-44fd-a2cc-830c44eea206 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:45.239265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 444/873] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:45 2022] GET /identity/v3/role_assignments?user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:45.418344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc08a6b-8b16-4879-8d79-01e6435f1d62 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.418717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc08a6b-8b16-4879-8d79-01e6435f1d62 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.419146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc08a6b-8b16-4879-8d79-01e6435f1d62 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.421716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 430/874] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:45.427918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.428221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.428504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.455538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:45.456757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:45.540124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 445/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:45.563698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc08a6b-8b16-4879-8d79-01e6435f1d62 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:45.688222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc08a6b-8b16-4879-8d79-01e6435f1d62 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:45.689890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:45.726633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:45.727811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.728295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.728911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.729587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:45.731068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:45.741022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:45.742067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:45.742994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 431/876] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:45.751261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:45.772224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:45.872375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca298c31-bd87-4253-bc2c-a9844f092808 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:45.914857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:45.916044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.916736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.917231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.918225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:45.918973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:45.921191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948da93-95b4-44a6-a6b6-703d39927f75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:45.927400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 446/877] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:45.934136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:45.989708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2cba-b4ac-4a41-917e-2aca7393a651 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.990320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2cba-b4ac-4a41-917e-2aca7393a651 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.990811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2cba-b4ac-4a41-917e-2aca7393a651 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:45.992724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 447/878] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:45.999362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:45.999657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:45.999943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:46.020993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:46.022368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:46.061778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d21808-149e-420f-b6c2-b7ea0f8a9859 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:46.102962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 448/879] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:46.121616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:46.126222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:46.128131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:46.128808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:46.129465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:46.131960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:46.132716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:46.135796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:46.138630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f8800-a3e0-4673-a254-283bff2f38aa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:46.147687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 432/880] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:46.246501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a40221-733c-41fb-9e74-af8aab4ab7f7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:46.288522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd73388-f540-4e1c-a701-9631cd3e238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:46.289189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd73388-f540-4e1c-a701-9631cd3e238f None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:46.289520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd73388-f540-4e1c-a701-9631cd3e238f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:46.289797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd73388-f540-4e1c-a701-9631cd3e238f None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:46.290564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd73388-f540-4e1c-a701-9631cd3e238f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:46.292078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd73388-f540-4e1c-a701-9631cd3e238f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:46.318882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 449/881] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:00:46 2022] POST /identity/v3/services => generated 269 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:47.214407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7b96-e37a-46b7-a7db-20964ed28277 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.214979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7b96-e37a-46b7-a7db-20964ed28277 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.214979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7b96-e37a-46b7-a7db-20964ed28277 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.216620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 433/882] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:47.221910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d4277-7617-46a9-8b3e-0a6bbf620f9d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.222345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d4277-7617-46a9-8b3e-0a6bbf620f9d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.222439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d4277-7617-46a9-8b3e-0a6bbf620f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.241090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-646d4277-7617-46a9-8b3e-0a6bbf620f9d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:47.242290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-646d4277-7617-46a9-8b3e-0a6bbf620f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:47.305639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:47.317472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 450/883] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:47.339439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf7b96-e37a-46b7-a7db-20964ed28277 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:47.373681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:47.455706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf7b96-e37a-46b7-a7db-20964ed28277 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:47.493524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:47.494200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.494482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.494752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.495450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:47.497027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:47.505533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c243c-dd86-4656-a96b-47d69b31dc49 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.506167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c243c-dd86-4656-a96b-47d69b31dc49 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.506644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c243c-dd86-4656-a96b-47d69b31dc49 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:47.507802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:47.508834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:47.509662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 434/884] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:47.511754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 451/885] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:47.516186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:47.517544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.518040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.518518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.537843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:47.539717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:47.622242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 452/886] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:47.633579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:47.640908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd73fe-d2af-4107-ad2d-63f6a490d060 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:47.646335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:47.679242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:47.680369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.680853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.681336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.682258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:47.682765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:47.684412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:47.690229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 435/887] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:47.696584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:47.765282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf45d6-237e-48b2-b7b0-37400e60c372 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:47.793649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:47.803447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:47.804100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.804417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.804695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.812677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5c14-8e1e-4301-bb4d-5cf8b086c4d1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:47.816143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:47.821303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:47.822311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:00:47.823556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:47.827147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 453/888] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:47.828339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:47.853821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:47.854782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.855280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.855697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.856612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:47.857180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:47.858994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:47.867098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 436/889] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:47.871640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:47.898822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:47.934567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252485b-ff76-4192-86a5-0eb1da51de35 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:47.970324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:47.970986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:47.971270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:47.971545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:47.972277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:47.972594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:47.974087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:47.983361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015344af-994b-42e0-b8a6-fee9c8dd3590 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:47.986985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 454/890] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:47.988872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.021404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.022423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.023027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.023503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.024540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] RBAC: Adding query filter params (interface=public, service_id=4f31274bda9f4f2aa48ef33802f9a338, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:48.025070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.026648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.037719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 437/891] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/endpoints?service_id=4f31274bda9f4f2aa48ef33802f9a338&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:48.039030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.041860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.104209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b37df-004c-49e4-9a10-9111b85f303b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.136631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.141417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.142111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.142406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.142645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.155123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.156800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.157887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6fea8-c34f-47bd-877f-0af333720711 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:48.173866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:48.174869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:48.175806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:48.175806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:48.175806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:48.175806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:48.180070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 455/892] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:48.181029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.201758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93c526-fd8b-414f-9e1d-4a8dc12bedb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.202862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c526-fd8b-414f-9e1d-4a8dc12bedb3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.203351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c526-fd8b-414f-9e1d-4a8dc12bedb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.203896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c526-fd8b-414f-9e1d-4a8dc12bedb3 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.204796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93c526-fd8b-414f-9e1d-4a8dc12bedb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.206437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93c526-fd8b-414f-9e1d-4a8dc12bedb3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.218963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 438/893] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:00:48.292838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398931-9dd3-4c75-b87e-7bf7f280bb36 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.293135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398931-9dd3-4c75-b87e-7bf7f280bb36 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.293325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398931-9dd3-4c75-b87e-7bf7f280bb36 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.294658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.298384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df8c0-31b0-4991-8ecb-1c41e92cfcf5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.299038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 439/894] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:48.300957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31604c65-d66e-43db-9b89-823105806f14 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.301229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31604c65-d66e-43db-9b89-823105806f14 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.301513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31604c65-d66e-43db-9b89-823105806f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.321654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-31604c65-d66e-43db-9b89-823105806f14 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:48.323265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-31604c65-d66e-43db-9b89-823105806f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:48.339613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.340257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.340543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.340816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.341539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:48.341845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.343267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.356079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 456/895] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:48.361841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.419363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 440/896] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:48.421125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14c91-9ae5-40f2-9722-d2acd52b5292 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.421423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14c91-9ae5-40f2-9722-d2acd52b5292 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.421707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14c91-9ae5-40f2-9722-d2acd52b5292 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.426859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 441/897] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:48.428276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.428558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.428847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.450793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:48.452063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:48.488249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36310-7bbf-4e2f-93e4-0ece2910f51e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.526197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 442/898] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:48.528366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.528868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.529374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.529662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.529946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.540100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.541572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.549945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:48.554083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:48.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 457/899] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:48.556873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.582385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.657495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a226675-5e84-4a05-8142-151a55b0ebd5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.679458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea824b2-99d2-4f56-b00f-b615a8d21f23 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.695730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.696386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.696675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.696954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.697676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.699114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.715163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 443/900] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:00:48 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:48.716008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.736063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.737217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.737707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.737987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.738727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.740277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:48.752490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:48.753502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:48.754334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 458/901] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:48.761119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.836364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.880433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb190123-58ea-4aae-8503-ea77ae0cbe15 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.884044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:48.907091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb0d29-0bda-48a4-91c6-59e83eccd1bb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:48.936636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.937676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.938160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.938518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.939369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:48.939682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.941174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.951544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:48.953149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:48.953440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:48.953722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:48.954423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:48.954733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:48.956143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:48.960774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 444/902] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:48.963140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 459/903] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:48.966552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:48.971506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:49.084449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fae07-fb05-4157-a557-108c6df88761 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.085254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124057-a03c-4f1d-9d68-a2b67183671d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.114417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:49.123376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.123629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.124027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.124317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.124436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.124596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.124882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.125242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.125368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:49.125648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.127062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.133844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 460/904] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:49.140882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:49.154000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:49.215104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.217904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.265536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091e5ff-8015-4050-9670-87652836eb8b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.307276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.308089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.308494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.309087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.310009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] RBAC: Adding query filter params (interface=public, service_id=c92eb84ac98749a59984a6067b461a91, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:49.310702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.312122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.319397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 461/905] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:00:49 2022] GET /identity/v3/endpoints?service_id=c92eb84ac98749a59984a6067b461a91&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:49.325827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:49.388958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 445/906] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 01:00:49.391724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:49.425301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:49.431453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:49.480953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e63f96-1f83-40c8-b306-f2323888bcbd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.531370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f521277-f088-4a7f-9649-d046d28dbe85 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.532558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.533235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.533510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.533785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.534826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.537454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.546879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 462/907] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:00:49 2022] GET /identity/v3/services => generated 1497 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:00:49.584136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.585286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.585804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.586321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.600322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.602440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:49.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:49.621993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:49.622912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:49.622912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:49.622912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:49.622912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:49.625564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 446/908] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:49.632051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:49.756465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8161dd8-bd02-4c82-9e58-a217781c2f16 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.794267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.795337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.795815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.796279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.797194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:49.797692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.799273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.802692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:49.814571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 463/909] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:49.819924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:49.908813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:49.930209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e7b8-bfe2-46cc-a664-985935e9001e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:49.966658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:49.967900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:49.968385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:49.968891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:49.978059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:49.979967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:49.993320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:49.994259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:49.995149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:49.997358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 447/910] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:50.003484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:50.111468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b553f8-17e2-4cbb-8553-bfbc7900a4f2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:50.148924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:50.150333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.150821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.151413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.152900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:50.153515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:50.155318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af700b4-9701-48b1-bffd-200fcff569e5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:50.169838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 464/911] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:50.176137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:50.283671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de4f4-d9e6-409c-ab7a-7b19c693b700 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:50.319160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1d6d1-8588-447d-8dd8-187f5058b035 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.319680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:50.319947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.320172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.320529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.320913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1d6d1-8588-447d-8dd8-187f5058b035 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.321335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] RBAC: Adding query filter params (role.id=cdb572ff146e4cf286b3ab8fa28c6a36, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:50.321619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1d6d1-8588-447d-8dd8-187f5058b035 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.321722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:50.323114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:50.323845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 465/912] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:50.334865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.334865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.334865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.335241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 448/913] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:00:50 2022] GET /identity/v3/role_assignments?user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=cdb572ff146e4cf286b3ab8fa28c6a36 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:50.350189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:50.351507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:50.410636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:50.431045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 466/914] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:50.454144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:50.458315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:50.570123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5bf1-1572-4035-9abb-7eca19e2cb13 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:50.608585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:50.609507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.609967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.610402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.611229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:50.612918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:50.623047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:50.624320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 01:00:50.625354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:50.627931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 449/915] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:50.639128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:50.743848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e4d0c-17b9-488f-b912-71d40f3d0b6b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:50.780397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:50.781437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.781962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.782455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.783387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:50.783917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:50.785496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d034e2c-fbf2-46f7-83b1-7e657b8bd839 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:50.791447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 467/916] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:50.800553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:50.871929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:50.907442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05122a-b6d0-4c63-82af-46899e6a2f31 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:50.930473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a604bf-68ff-49ff-97a9-e27834dd8327 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.931174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a604bf-68ff-49ff-97a9-e27834dd8327 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.931656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a604bf-68ff-49ff-97a9-e27834dd8327 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.933221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 468/917] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:50.938540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.939144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.939626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.944966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:50.945668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:50.945953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:50.946229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:50.946947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:50.947356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:50.949208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:50.954170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 450/918] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jul 28 01:00:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:50.959949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:50.960491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:50.961939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:51.011790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:51.042878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 469/919] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:51.063121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:51.083632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf5377-97a0-435e-aba3-c5251fe84d57 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:51.115716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:51.122527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:51.123201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.123563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.123761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.124506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:51.125964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:51.154861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 451/920] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:00:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:51.188767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d47e-c839-4738-9752-f7f11a5918c1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:51.236858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:51.237677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.237969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.238262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.239116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:51.240695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:51.251604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:51.252676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:51.253504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 470/921] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:51.262178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:51.377108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0f0ef-d2ce-4872-b0a6-4fb5208c4c5e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:51.414181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:51.415242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.415742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.416212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.417102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:51.417629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:51.419246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:51.425281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 452/922] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:51.431659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:51.481327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:51.559918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1cf46-3684-4afe-ba34-965ae2fd9ec7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:51.598919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:51.599561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.599846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.600122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.600824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:51.601132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:51.602572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0855fe7-7b2f-4d1f-986e-7c280442a707 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:51.608483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 471/923] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:51.614174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:51.673281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:51.730353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce5a2a-e3b2-4ec7-974a-7672b94f64db None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:51.766744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:51.767670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.768103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.768523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.769458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] RBAC: Adding query filter params (interface=public, service_id=c3c96bc6e2c3428c92e03fd717c4c415, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:51.769942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:51.771520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:51.776465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c584f-02f1-4bf2-80ee-144b8e5d0d7d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.776755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c584f-02f1-4bf2-80ee-144b8e5d0d7d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.777042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c584f-02f1-4bf2-80ee-144b8e5d0d7d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.778176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 453/924] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:00:51 2022] GET /identity/v3/endpoints?service_id=c3c96bc6e2c3428c92e03fd717c4c415&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:51.778573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 472/925] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:51.783878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.784171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.784451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.787779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:51.804992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:51.806576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:51.895051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 473/926] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:51.919214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:51.920549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9453b5-254a-4b28-a23d-a3383e4265ea None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:51.940077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:51.962645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:51.963313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:51.963593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:51.963871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:51.964538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:51.965984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:51.974251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 454/927] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:00:51 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:00:51.993020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:52.074021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c3e4b-7f27-4592-97fd-cee1c2b085f0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:52.114082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:52.115138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.115626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.116102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.116919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:52.118653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:52.129177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:52.130742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 01:00:52.131645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:52.132870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 474/928] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jul 28 01:00:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:52.139679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:52.263375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c9e9-acb7-420d-b3d7-c5b7681b2b90 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:52.299146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:52.299785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.300069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.300372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.301335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:52.301840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:52.303614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:52.315292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 455/929] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:52.320912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:52.457569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b17f2-a12a-4b7f-9112-83645c3fac1b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:52.467541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:52.496570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:52.497209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.497577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.497849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.498578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:52.498914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:52.501662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213600fd-3f4f-41dc-973f-44ca0e3daeaf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:52.510664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 475/930] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:52.516922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:52.560332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe6c9-89ae-4a50-9003-1faa08fb6e0b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.561305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe6c9-89ae-4a50-9003-1faa08fb6e0b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.561305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe6c9-89ae-4a50-9003-1faa08fb6e0b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.562546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 476/931] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:52.574924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.575257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.575511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.587903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:52.601600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:52.603076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:52.655686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6afa1b-6b64-4498-b587-db0b05d550ec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:52.689030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 477/932] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:52.696414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:52.697077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.697354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.697622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.698382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:52.699846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:52.716432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:52.734523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 456/933] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:00:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:52.839936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:52.851201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176d935-5eba-41a5-b756-5281c4fc5b1c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:52.898301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:52.899368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:52.899804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:52.900245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:52.911628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:52.916873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:52.917987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:00:52.919018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:52.921402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 478/934] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 01:00:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:52.928602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.037671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:53.042685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8dec4-69e2-4ed7-9949-0188b1a54c87 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.079660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.080610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.081101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.081543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.082445 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:53.082952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.084525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.094437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 457/935] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:00:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:53.100425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.224541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11505131-4baf-4c0d-a25f-352945c3f22d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.264695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.265676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.266160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.266613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.279240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.280979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.290884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c903c33c-1c24-4e74-9128-b8b4fe01162d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:53.298519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:53.299598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:53.300511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:53.300511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:53.300511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:53.300511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.302532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 479/936] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:53.308431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.400101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd3906-e8df-4b5f-a6d2-1041798d6653 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.400471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd3906-e8df-4b5f-a6d2-1041798d6653 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.400840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd3906-e8df-4b5f-a6d2-1041798d6653 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.402555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 480/937] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:53.407423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.407800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.408187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.413793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee124e-4535-4482-b122-9761b4008940 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.427993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:53.429438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:53.451781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.452693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.453115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.453527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.454382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:53.454825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.456404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.470212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 458/938] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:53.476960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.503453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:53.523181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 481/939] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:53.543545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.586627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a8490-42a2-4d02-827b-dda694280022 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.622147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.623276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.623702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.624137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.632662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.634500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:53.647213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:53.648194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:53.649200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:53.656454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 459/940] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:53.657052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee6f87-8803-448a-b1e2-a1edc352f2a1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.658235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.687946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:53.696721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.697383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.697752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.698827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.698827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.700304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.703098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING keystone.server.flask.common [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 01:00:53.724773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.sql.core [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 01:00:53.724773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 01:00:53.724773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [parameters: {'id': '3cfdccdce0ea46b2a4a78a07ddc0459e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 01:00:53.724773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80415) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3cfdccdce0ea46b2a4a78a07ddc0459e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3cfdccdce0ea46b2a4a78a07ddc0459e. Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application conn.query(q) Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 01:00:53.743253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application result.read() Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 01:00:53.744223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application next(self.gen) Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application yield resource Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application next(self.gen) Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise self.value Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application session.commit() Jul 28 01:00:53.745766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self.session.flush() Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application compat.raise_( Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 01:00:53.746664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 01:00:53.748034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application conn.query(q) Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application result.read() Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 01:00:53.750309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [parameters: {'id': '3cfdccdce0ea46b2a4a78a07ddc0459e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.752422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:53.754279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3cfdccdce0ea46b2a4a78a07ddc0459e. Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 482/941] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jul 28 01:00:53 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:53.758478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.771311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:53.798154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9c0f0-1bcd-4b04-a5c7-6d333b757d7f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.851237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.851914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.852200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.852473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.853284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:53.853591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.855055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.882922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 460/942] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:53.885292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:53.893553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e035f-bc3b-4ff4-85b8-17e8bbaf0403 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:53.943186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:53.943935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:53.944218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:53.944893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:53.954300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:53.956513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:53.972350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:53.973310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:53.974191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:53.979459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 483/943] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:53.980902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.011367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779f89-9d51-43d7-b616-a264dc1b43ce None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.038900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.057772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.058436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.058991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.059290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.071230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.075969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.078909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:54.100210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:54.101450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:54.102635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:54.102635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:54.102635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:54.102635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:54.109273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 461/944] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:54.110365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.113932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf94b7-a3a7-4107-b629-f74965a66f28 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.156285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.157080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.157458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.157831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.158670 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:54.159093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.160629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be70b8f-5aa7-419c-a47c-8ced3226ba17 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.188740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 484/945] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:54.188740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4e00-25c1-45bb-bb98-7484cc3b8659 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.188740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4e00-25c1-45bb-bb98-7484cc3b8659 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.188740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4e00-25c1-45bb-bb98-7484cc3b8659 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.188740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 485/946] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:54.191071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.191464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.191839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.211967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:54.213625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:54.239033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab085c-735f-4efc-9705-2ec87677bbc7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.279878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.280533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.280821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.281101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.281823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:54.282137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.283756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.303602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 486/947] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:54.308050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 462/948] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:54.313813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.319535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.368150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.393030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.456175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef03ab-bca1-4ace-bc1a-6e101118d3ae None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.457397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8f29-09f0-4c0b-bf5e-65c8c0d454d2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.503020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.503563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.503847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.504113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.507181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.508228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.508605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.509023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.509413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.509790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.517823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.519680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:54.519903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:54.521026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:54.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 463/949] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:54.528985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:54.539495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:54.540779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:54.541703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:54.544789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 487/950] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:54.550766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.640468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6410-74a5-4bcd-8261-ac77ccea92cf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.658390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.671576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.678730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.679490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.679951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.680322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.681120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:54.681819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.683141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.686649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802660f5-15f3-455b-b7de-9c30129a60e5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.693309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 464/951] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:54.694486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.733027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341178-b438-431d-a1b0-db99af39478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.733906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341178-b438-431d-a1b0-db99af39478b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.734329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341178-b438-431d-a1b0-db99af39478b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.734761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341178-b438-431d-a1b0-db99af39478b None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.735701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341178-b438-431d-a1b0-db99af39478b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:54.736168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341178-b438-431d-a1b0-db99af39478b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.737867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341178-b438-431d-a1b0-db99af39478b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.756905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 488/952] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:54.758352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341178-b438-431d-a1b0-db99af39478b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.813470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909d58-d5aa-4a99-911f-767a8bc79cdc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.861356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.861994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.862312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.862587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.863346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:54.863669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.865063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e250b5d-4f85-4bfa-9fe5-574fa4e22988 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.870792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 465/953] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:54.872230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d9db-96d6-4f6e-b2e5-a08399dc0d54 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.872516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d9db-96d6-4f6e-b2e5-a08399dc0d54 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.872795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d9db-96d6-4f6e-b2e5-a08399dc0d54 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.874329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 466/954] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:54.876493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341178-b438-431d-a1b0-db99af39478b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:54.877369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d9db-96d6-4f6e-b2e5-a08399dc0d54 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:54.915545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7df46-df53-4a77-8c1f-5743289fb79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:54.916531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7df46-df53-4a77-8c1f-5743289fb79c None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.916957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7df46-df53-4a77-8c1f-5743289fb79c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.917373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7df46-df53-4a77-8c1f-5743289fb79c None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8d5e9ee4b7f94c08bfb5eeb391bfa700/roles/cdb572ff146e4cf286b3ab8fa28c6a36` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.919892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7df46-df53-4a77-8c1f-5743289fb79c None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=8d5e9ee4b7f94c08bfb5eeb391bfa700, role_id=cdb572ff146e4cf286b3ab8fa28c6a36)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:54.921621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7df46-df53-4a77-8c1f-5743289fb79c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:54.932407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:54.938251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 489/955] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:00:54 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8d5e9ee4b7f94c08bfb5eeb391bfa700/roles/cdb572ff146e4cf286b3ab8fa28c6a36 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:00:54.939729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:54.940163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:54.940579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:54.962529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:54.964127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:54.977740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:55.038647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d9db-96d6-4f6e-b2e5-a08399dc0d54 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:55.052308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 490/956] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:55.074349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:55.086363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:55.087422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:55.087858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:55.088845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:55.090103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:55.092165 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:55.130879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 467/957] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:00:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:55.216206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150aaf7-1efb-4134-b45c-f929c9d1994c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:55.265923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:55.266758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:55.267207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:55.267574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:55.268322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:55.269840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:55.280417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:55.281776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:00:55.282890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:55.285886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 491/958] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:55.291434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:55.402806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:55.407396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637a47c-1d91-42d4-94cb-8881f46b2194 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:55.453066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:55.453879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:55.454265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:55.454657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:55.455548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:55.455998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:55.457487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13736e67-72ac-4253-bf22-fe7b38bb0d07 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:55.463815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 468/959] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:55.470196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:55.485809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:55.593872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2ebdb-50fd-4114-b76e-abd803e6e9f2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:55.634619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:55.635301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:55.635586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:55.635870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:55.636572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:55.636880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:55.638378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3221a9c-a94f-4053-8aae-c6a86d08ead3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:55.644069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 492/960] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:00:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:56.407227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d855-3f58-4f1c-98ba-0d019359eb20 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:56.407830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d855-3f58-4f1c-98ba-0d019359eb20 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:56.407830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d855-3f58-4f1c-98ba-0d019359eb20 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:56.409315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 469/961] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:56.416508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:56.417426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:56.418184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:56.443599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:56.445757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:56.546213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:56.553060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 493/962] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:56.578624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d855-3f58-4f1c-98ba-0d019359eb20 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:56.693336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d855-3f58-4f1c-98ba-0d019359eb20 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:56.732324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:56.732974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:56.733258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:56.733531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:56.734244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:56.735749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:56.738401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING keystone.server.flask.common [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 01:00:56.753731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 01:00:56.753731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 01:00:56.753731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [parameters: {'user_id': 'c1edd724b9954c648adf0a817c9bef18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 01:00:56.753731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 28 01:00:56.765449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 01:00:56.766380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 01:00:56.767382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 01:00:56.768279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 01:00:56.769184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 01:00:56.770036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c1edd724b9954c648adf0a817c9bef18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:56.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 01:00:56.773142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:56.774284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 470/963] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jul 28 01:00:56 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:56.779066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:56.906482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75993dfa-8219-4ea1-a26c-f91d39300fab None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:56.947748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:56.948342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:56.948629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:56.948904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:56.957603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:56.959136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0c3a0c18-6953-4dd1-8ce0-d311ee2aeedf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:56.972571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:56.974475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:56.975751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 494/964] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:00:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:56.980526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.013438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:57.096723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9ab29-735e-4fd9-a65a-272beffb6d11 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:57.134324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:57.135646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.136153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.136619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.137591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:57.138093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:57.139978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:57.180971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7dbf2-8501-4f8c-8dc5-98ec4dc33f6e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.181321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7dbf2-8501-4f8c-8dc5-98ec4dc33f6e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.181546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7dbf2-8501-4f8c-8dc5-98ec4dc33f6e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.183262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 495/965] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:00:57.183793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 471/966] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jul 28 01:00:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:57.191970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdae0e1-6964-4526-b01b-44795a302db4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.191970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdae0e1-6964-4526-b01b-44795a302db4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.191970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdae0e1-6964-4526-b01b-44795a302db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.209480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7fdae0e1-6964-4526-b01b-44795a302db4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:57.211273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7fdae0e1-6964-4526-b01b-44795a302db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:57.298449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:57.314211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 496/967] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:57.345318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.384142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54243e96-2f36-49d2-b1c4-96004117a76c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.384594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54243e96-2f36-49d2-b1c4-96004117a76c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.384858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54243e96-2f36-49d2-b1c4-96004117a76c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.386483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 497/968] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:57.394993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.395623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.396171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.425940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:57.427311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:57.481618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:57.510670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4036d-bee3-41b6-9d49-50d32caa0a31 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:57.529275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 498/969] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:00:57.551739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.566213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:57.566981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.567359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.567641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.580982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:00:57.586512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:57.588755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:57.590668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 472/970] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 01:00:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:57.595971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.686967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca213bf0-2b43-4a4c-babd-d1d323fc2660 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:57.718749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:57.728604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce82b-b03b-4ee7-8521-3057cd8de44b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:57.741168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:57.742058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.742389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.742667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.743366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:57.745392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:57.760205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:57.762246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:57.763133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 499/971] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:00:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:00:57.768115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.776572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:57.777312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.777596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.777876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.778612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:57.778950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:57.780372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:57.790503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 473/972] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:00:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:57.798479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.814036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:57.921165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6260325-8cab-487e-897f-83bd54b526d8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:57.934693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c7f58-e0be-45bc-93ee-09184755a65c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:57.961318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:57.961969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.962256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.962533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.963271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:57.963586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:57.965006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3b1d0-07cf-461e-b00b-838191bd5215 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:57.970602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 500/973] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:57.972026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67885a8-cc2c-4fbd-b576-222ff545035a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.972328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67885a8-cc2c-4fbd-b576-222ff545035a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.972609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67885a8-cc2c-4fbd-b576-222ff545035a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:57.973965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 501/974] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:00:57.979756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67885a8-cc2c-4fbd-b576-222ff545035a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:57.993112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:57.993707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:57.993999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:57.994286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.007097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.008874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-da10b5cd-63c5-4f35-a023-9fc27aa0a30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:58.024979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:58.026959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 474/975] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.028731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.049659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:00:58.050229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:58.051312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:00:58.116888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:58.131679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67885a8-cc2c-4fbd-b576-222ff545035a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.136061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 475/976] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:00:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:00:58.136964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.177527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.178172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.178468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.178804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.179469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:58.179924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.181331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.186677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 502/977] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:00:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:58.187345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.265139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fb603-95ef-4065-b334-d5ab979a8603 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.297525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adb5f8-1afc-4a57-a43e-393dbd22ac10 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.302406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.303077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.303367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.303645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.304359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:58.304743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.306129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.317469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:58.318295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 476/978] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:58.319190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.345305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.345946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.346221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.346502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.347255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.348642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.365637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 503/979] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:00:58 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:00:58.366483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.384911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:58.460954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8de82-17c4-4a16-9e17-f3604f0072fc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.518925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.520315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.520605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.520887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.521695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] RBAC: Adding query filter params (interface=admin, service_id=c3c96bc6e2c3428c92e03fd717c4c415, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:58.522021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.523573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.529651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb79842-5ac7-4828-aa5a-49fd2b41f299 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.530894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 477/980] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jul 28 01:00:58 2022] GET /identity/v3/endpoints?service_id=c3c96bc6e2c3428c92e03fd717c4c415&interface=admin®ion_id=RegionOne => generated 193 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:58.536628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.575674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.576488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.576864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.577246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.585858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.587372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:00:58.603226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:58.604345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:58.606759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 504/981] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:00:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:58.610205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.652357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:58.692224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57fe06-94b6-49c2-98c9-87e34c8f67a7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.707257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:58.738616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dda7e0-616e-4132-9a97-f2dd006a366c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:58.739247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.739919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.740210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.740491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.741163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.742879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.758869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 478/982] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:00:58 2022] GET /identity/v3/services => generated 1727 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:00:58.792504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:58.793224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:58.793580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:58.793926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:58.794860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:58.795253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:58.798932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:58.838793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 505/983] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jul 28 01:00:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:00:58.840358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:58.976689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866b47a-560e-4f53-8822-39cf0105d142 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:59.020705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:59.021855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:59.021855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:59.022023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:59.077407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:59.111506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:59.113281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:59.238882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 479/984] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:00:58 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:00:59.240659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:59.349624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:59.361476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefd53b-19b5-452f-b53d-927d5270ee4c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:59.398382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:59.399226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:59.399515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:59.399818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:59.410091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:59.411583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:00:59.427142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:59.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:59.428964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:00:59.428964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:00:59.428964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:00:59.428964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:59.431164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 506/985] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:59.434306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:59.528110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:59.569378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f5d51-7cda-43ae-aa29-07dee8763a36 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:59.620982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:59.621980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:59.622476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:59.623006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:59.624314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:59.624868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:59.627503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:59.655093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 480/986] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:00:59.660107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:59.765359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c7dfd-5545-42b9-ad0c-c1371f407176 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:59.807854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:59.808515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:59.808809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:59.809082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:59.816131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:59.817571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:00:59.833409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:00:59.834300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:00:59.840512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 507/987] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:00:59.841005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:00:59.943878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe62cb-cb0e-4c40-a34f-914172062e75 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:00:59.974066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:00:59.980355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:00:59.980998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:00:59.981311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:00:59.981593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:00:59.982361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:00:59.982678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:00:59.984161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdbf99-a8a1-43cd-9d93-6b15caf37d32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:00:59.997367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 481/988] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:00.003495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:00.006920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:00.117028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4fe5-0cd3-4d93-b487-d4a179a50825 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:00.153479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:00.154118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.154403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.154672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.155504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] RBAC: Adding query filter params (role.id=cdb572ff146e4cf286b3ab8fa28c6a36, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:00.155818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:00.157239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfd09c-536d-406d-8660-b0807701a6d9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:00.163906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 508/989] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:00 2022] GET /identity/v3/role_assignments?user.id=8d5e9ee4b7f94c08bfb5eeb391bfa700&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=cdb572ff146e4cf286b3ab8fa28c6a36 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:00.544234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aaf7b-dac2-485f-bba1-15d800aed22d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.544981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aaf7b-dac2-485f-bba1-15d800aed22d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.544981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aaf7b-dac2-485f-bba1-15d800aed22d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.546388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 482/990] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:01:00.551695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.552219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.552327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.571570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:00.572777 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:00.643407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:00.651856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 509/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:00.676637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aaf7b-dac2-485f-bba1-15d800aed22d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:00.788088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aaf7b-dac2-485f-bba1-15d800aed22d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:00.823047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:00.823709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.823991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.824268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.824910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:00.826281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f69efce7-e858-44f3-821d-bffc56554675 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:00.835808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:00.836805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:01:00.837642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:00.839171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 483/992] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:01:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:00.843137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:00.945775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5a204-4b92-48db-96d1-d040fb906902 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.946387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5a204-4b92-48db-96d1-d040fb906902 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.946745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5a204-4b92-48db-96d1-d040fb906902 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.948356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 484/993] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:00.949645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80324c4-0592-41da-8082-46f2344fc6d5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:00.953812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.954410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.954752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.975301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:00.976596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:00.991901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:00.992886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:00.993365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:00.993714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:00.994396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:00.994771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:00.996194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.006743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 510/994] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:01:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:01.008456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.031125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:01.054958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 485/995] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:01.077212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.084450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:01.130439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337ae91-5a8c-4291-bb72-201886ce1f57 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.167752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.168400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.168687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.168965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.169666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:01.169979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.171449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.176984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 511/996] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:01:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:01.183358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.193238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4980-d2d1-40ec-86f5-4e4973562429 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.234367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.235068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.235397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.235631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.236282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.237670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:01.247766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:01.248824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:01.249657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 486/997] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jul 28 01:01:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:01.255908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.299938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7b2bb-fc79-4c4c-b9ce-8201225ef05a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.317291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:01.331427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:01.348627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.349337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.349576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.350487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.351339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] RBAC: Adding query filter params (interface=public, service_id=ce9f8d0393144cc3ba4f041f9c729df0, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:01.351667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.353568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.362484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 512/998] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:01:01 2022] GET /identity/v3/endpoints?service_id=ce9f8d0393144cc3ba4f041f9c729df0&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:01.367654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.382644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a958e5-24a9-4694-8a4f-6641b678818d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.421053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.421703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.421993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.422276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.423315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:01.423634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.425026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.430393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 487/999] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:01:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:01.436235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.512720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d97f3-245a-4b41-88e1-3815ed9bcf64 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.564515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2a8d3-e5c3-47d0-a9f1-490e84cbfed1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.565421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.565421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.565867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.565939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.566596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.568131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.575067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 513/1000] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:01:01 2022] GET /identity/v3/services => generated 1727 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:01:01.582707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:01.613129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.613974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.614385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.614799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.615726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:01.616187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.617809 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b2532-d238-4779-b6ab-c5ae437e8bf0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.629251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 488/1001] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jul 28 01:01:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:01.631664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:01.696155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:01.755131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261957c-c039-4233-88bc-a71ec661039d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:01.792551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527d9d9-2502-4961-9e28-5de0ca3a9dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:01.793198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527d9d9-2502-4961-9e28-5de0ca3a9dc5 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:01.793482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527d9d9-2502-4961-9e28-5de0ca3a9dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:01.793757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527d9d9-2502-4961-9e28-5de0ca3a9dc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:01.794501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527d9d9-2502-4961-9e28-5de0ca3a9dc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:01.795927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527d9d9-2502-4961-9e28-5de0ca3a9dc5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:01.814962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 514/1002] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:01:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:02.215873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f915e9-35c2-4444-82e3-30b9c68d0f4a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:02.216152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f915e9-35c2-4444-82e3-30b9c68d0f4a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:02.216529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f915e9-35c2-4444-82e3-30b9c68d0f4a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:02.218092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 489/1003] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:02.223639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:02.223925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:02.224202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:02.243285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:02.244483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:02.325883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 515/1004] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:02.348073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f915e9-35c2-4444-82e3-30b9c68d0f4a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:02.470055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f915e9-35c2-4444-82e3-30b9c68d0f4a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:02.510962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:02.511625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:02.511909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:02.512186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] PATH_INFO: `/v3/services/image` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:02.512831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:02.514220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:02.523509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:02.524602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:02.525432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:02.527048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 490/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 01:01:02 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:02.530818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:02.577314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:02.658118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c9054-534e-487b-ae94-0a859685a317 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:02.697207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:02.697850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:02.698134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:02.698413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:02.699136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] RBAC: Adding query filter params (name=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:02.699449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:02.700823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6f350-ca18-4ca8-949c-17e8f791b568 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:02.707019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 516/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:02.711976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:02.763066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:02.825372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771d240-cfb1-40bd-9888-66ba605e0382 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:02.860506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:02.861143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:02.861472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:02.861754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:02.862449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] RBAC: Adding query filter params (type=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:02.862756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:02.864177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279b47d-c4b2-4418-911e-db3e26867180 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:02.869060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 491/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:03.627307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6f005-f338-4015-ad53-d1b7b2cc6406 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:03.627924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6f005-f338-4015-ad53-d1b7b2cc6406 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:03.627924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6f005-f338-4015-ad53-d1b7b2cc6406 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:03.629216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 517/1008] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:03.635335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba339a-0eb9-4595-b85c-9159e331321b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:03.635703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba339a-0eb9-4595-b85c-9159e331321b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:03.636058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba339a-0eb9-4595-b85c-9159e331321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:03.655105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0eba339a-0eb9-4595-b85c-9159e331321b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:03.656368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0eba339a-0eb9-4595-b85c-9159e331321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:03.737285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 492/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:03.760143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6f005-f338-4015-ad53-d1b7b2cc6406 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:03.839823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c6c45-7a16-4a75-9382-be0a8e22323c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:03.840097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c6c45-7a16-4a75-9382-be0a8e22323c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:03.840374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c6c45-7a16-4a75-9382-be0a8e22323c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:03.841755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 493/1010] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:03.846859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:03.847156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:03.847436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:03.865649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:03.866824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:03.894710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6f005-f338-4015-ad53-d1b7b2cc6406 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:03.896371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:03.901167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:03.933906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:03.934567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:03.934955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:03.935186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:03.935805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:03.937183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:03.942692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 494/1011] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:03.947936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:03.949018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 01:01:03.949842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:03.954533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 518/1012] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:01:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:03.955404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:03.966170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:04.065275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc2b0b-72c4-4791-96b0-1e7fa4c23ff4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:04.083379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93789537-16c0-44f8-972a-720c97e4eff9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:04.104806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:04.105930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.106214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.106490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.107220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:04.107526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:04.110451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:04.118470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 495/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:01:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:04.122179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:04.123091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.123378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.123655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.124199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:04.125020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:04.126469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:04.128263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 519/1014] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:04.134826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:04.158370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:04.175637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:04.234185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ebd9d-0138-485a-aaae-6a97b858b9d7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:04.248462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e521ef9-9570-4756-8bd8-5cbf0f9514ce None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:04.271376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:04.272021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.272308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.272589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.273359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:04.273674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:04.275089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:04.283027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 496/1015] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jul 28 01:01:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:04.286347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:04.291658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a9940-0f38-4a8b-a475-6cf2ccb06a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:04.292308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a9940-0f38-4a8b-a475-6cf2ccb06a21 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.292599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a9940-0f38-4a8b-a475-6cf2ccb06a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.292880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a9940-0f38-4a8b-a475-6cf2ccb06a21 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.293580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a9940-0f38-4a8b-a475-6cf2ccb06a21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:04.295089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a9940-0f38-4a8b-a475-6cf2ccb06a21 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:04.333821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 520/1016] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:01:04 2022] POST /identity/v3/projects => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:01:04.400741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:04.411060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309df05-a15f-4971-96e6-8269fd873a2d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:04.453841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68da323-5c90-420f-b376-7d1f9057946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:04.454560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68da323-5c90-420f-b376-7d1f9057946e None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.454867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68da323-5c90-420f-b376-7d1f9057946e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.455143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68da323-5c90-420f-b376-7d1f9057946e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.455884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68da323-5c90-420f-b376-7d1f9057946e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:04.457315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68da323-5c90-420f-b376-7d1f9057946e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:04.482537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 497/1017] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:01:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:01:04.897321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f8147-dd7a-40ee-b90d-c5d85420c9f3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.897924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f8147-dd7a-40ee-b90d-c5d85420c9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.897924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f8147-dd7a-40ee-b90d-c5d85420c9f3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.899455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 521/1018] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:04.905550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340b60c-8d43-40c8-a3d2-8b0e6de8ddf2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:04.906213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340b60c-8d43-40c8-a3d2-8b0e6de8ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:04.906213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340b60c-8d43-40c8-a3d2-8b0e6de8ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:04.925761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5340b60c-8d43-40c8-a3d2-8b0e6de8ddf2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:04.927146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5340b60c-8d43-40c8-a3d2-8b0e6de8ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:05.009106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 498/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:05.032122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f8147-dd7a-40ee-b90d-c5d85420c9f3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:05.125441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:05.143171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f8147-dd7a-40ee-b90d-c5d85420c9f3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:05.178667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b42c4e-6e89-4170-acc8-f6a41d63193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:05.179255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b42c4e-6e89-4170-acc8-f6a41d63193a None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:05.179614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b42c4e-6e89-4170-acc8-f6a41d63193a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:05.179875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b42c4e-6e89-4170-acc8-f6a41d63193a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:05.180558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b42c4e-6e89-4170-acc8-f6a41d63193a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:05.181930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b42c4e-6e89-4170-acc8-f6a41d63193a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:05.204049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 522/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:01:05 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:06.416032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3117e5e-5340-4561-9a0b-d98367ec04f1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.416634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3117e5e-5340-4561-9a0b-d98367ec04f1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.416634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3117e5e-5340-4561-9a0b-d98367ec04f1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.418191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 499/1021] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:06.423447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f2886-66a4-4292-96fa-7f19f4809832 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.423901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f2886-66a4-4292-96fa-7f19f4809832 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.423990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f2886-66a4-4292-96fa-7f19f4809832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.442476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fc6f2886-66a4-4292-96fa-7f19f4809832 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:06.443708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fc6f2886-66a4-4292-96fa-7f19f4809832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:06.534896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 523/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:06.555190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3117e5e-5340-4561-9a0b-d98367ec04f1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:06.605359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bbb06-6d05-4081-a348-16a4f3ca0d9e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.605831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bbb06-6d05-4081-a348-16a4f3ca0d9e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.606124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bbb06-6d05-4081-a348-16a4f3ca0d9e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.607671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 524/1023] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:06.614062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.614391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.614631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.639809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:06.641007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:06.648001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:06.677385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3117e5e-5340-4561-9a0b-d98367ec04f1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:06.715911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:06.716872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.717153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.718871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.718986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 525/1024] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:06.719089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:06.720263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:06.722884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 500/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:06.727650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:06.730478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:06.740989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:06.841706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b073f6-ec58-4984-83af-b70feab85faa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:06.860861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e7b2d-9735-489b-9457-8d77ebdf39ab None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:06.879962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:06.880884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.881343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.881792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.882541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:06.884263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:06.896426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:06.897091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:06.897393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:06.897649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:06.908645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 526/1026] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:01:06 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:06.920330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:06.926311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:06.927763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:01:06.928966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:06.931306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 501/1027] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 01:01:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:06.933903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.003879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.054466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee927ea-41ca-4210-80e5-c55cde862d9a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.095645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.096312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.096604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.096884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.097637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:07.097947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.099375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852d498-c2a6-4ca3-90a5-9bab10fe746e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.108762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 527/1028] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:01:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:07.114712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.120174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.227891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868275c-432f-4198-a589-af3c8ca2991c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.245108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a8d2d-24f0-47be-b2bf-65e70cfcd65a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.245418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a8d2d-24f0-47be-b2bf-65e70cfcd65a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.245668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a8d2d-24f0-47be-b2bf-65e70cfcd65a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.247341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 528/1029] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:01:07.252220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.252612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.252781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.263208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.263876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.264145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.264437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.274282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:07.275816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:07.277591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.279064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:07.295139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:07.296102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:07.297006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:07.297006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:07.297006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:07.297006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:07.301041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 502/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:07.301988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.351173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 529/1031] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:07.361746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.373234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.408826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78596124-e70b-4f1a-99a0-5f1e493659a0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.424329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.447039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.448344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.448771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.449272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.450443 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:07.451031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.453051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.469656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 503/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:07.475399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.490469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4d245-8db5-4f79-ad7f-33df9f948437 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.527156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.527822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.528119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.528390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.529030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.530393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:07.539566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:07.540594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:07.541431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 530/1033] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:07.548707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.597559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53ad0c-664c-4194-9fb7-cd74da1e7fbf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.615712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.637233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.637896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.638171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.638442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.646243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.647756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:01:07.659633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:07.660661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:07.661618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 504/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:07.664296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ea289-081e-4686-b9e6-435635a869b6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.666565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.679796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.704591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.705232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.705514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.705788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.706478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] RBAC: Adding query filter params (name=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:07.706783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.708182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.718783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 531/1035] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:07.719991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.784762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740186-492d-43a9-a098-1a91f980dea7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.821566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.822284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.822471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.822764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.823563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:07.823888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.825315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.830546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e37ef-8381-40a4-bc1a-9a5e3b430e01 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.844056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 505/1036] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:07.849727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.864848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.865466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.865751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.866026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:07.866712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] RBAC: Adding query filter params (type=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:07.867116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:07.868517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:07.873312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 532/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:07.880274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:07.886529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.933439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:07.960459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda217ae-1488-4c35-b6fc-99dacf1c42a8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.996285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d470e-c7e9-4c67-ad1d-8fe8e57cb84e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:07.997422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:07.998050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:07.998342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:07.998608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:08.034506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:08.034944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:08.035172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:08.035444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:08.036201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] RBAC: Adding query filter params (interface=public, service_id=31b6597715894ee281cc2271b2b9013f, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:08.036511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:08.037869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:08.044414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 533/1038] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/endpoints?service_id=31b6597715894ee281cc2271b2b9013f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:08.050015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:08.074310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:08.076478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:08.115524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:08.160908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d57370-b640-4f86-a357-29e906591d33 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:08.169132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:08.198343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:08.199014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:08.199291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:08.199564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:08.200234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:08.201586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:08.209024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 534/1039] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:01:08 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:01:08.209663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 506/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 01:01:08.215517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:08.343679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e4aa4-02d8-407e-bba8-15415f230468 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:08.384551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:08.385285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:08.386255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:08.386255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:08.397523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:08.399317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:08.414354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:08.415582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:08.416761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:08.416761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:08.416761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:08.416761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:08.416761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 535/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:08.421623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:08.536997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911bf88-7eff-40ff-ad5f-4460c9eaec94 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:08.572379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:08.573073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:08.573357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:08.573630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:08.574393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:08.574760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:08.576185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:08.577944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:08.597594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 507/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:08.601850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:08.638143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:08.716994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634191-045d-46e4-bb0e-cf88fb3d3cd2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:08.765261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:08.766383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:08.766927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:08.767421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:08.779635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:08.782224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d9970255-0c4d-40b3-a19c-e249f91c9ac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:01:08.803951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:08.805034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:01:08.805922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:08.806982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 536/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:08.815697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:08.956918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b054fe1-06b8-4229-a650-89c70fc71202 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:09.019269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:09.020105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0176c-d7d8-4e51-ba5c-122e46101988 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.020284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.020405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0176c-d7d8-4e51-ba5c-122e46101988 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.020671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0176c-d7d8-4e51-ba5c-122e46101988 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.020768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.021206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.022175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 537/1044] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:01:09.022618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:09.023338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:09.026041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:09.027078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.027371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.027653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.045680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 508/1045] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:09.047699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:09.049716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:09.051688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:09.111280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:09.164701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 538/1046] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:09.174140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87762d9-15fa-45de-aa69-78bbff6a91de None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:09.187522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:09.203464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:09.213552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:09.214118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.214377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.214868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.215513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] RBAC: Adding query filter params (role.id=cdb572ff146e4cf286b3ab8fa28c6a36, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8f2f77d4f4c44890ac1ab88ff66100bd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:09.215831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:09.217379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:09.234875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 509/1047] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:09 2022] GET /identity/v3/role_assignments?user.id=8f2f77d4f4c44890ac1ab88ff66100bd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=cdb572ff146e4cf286b3ab8fa28c6a36 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:09.314193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa220-0bb9-4f68-8f82-408ea5397fd4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:09.354207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:09.354910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.355204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.355482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.367760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:09.372949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:09.374023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:09.380399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 539/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:09.381156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:09.497427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf1dc-6959-4e7d-89a4-7597bed0c945 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:09.520047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:09.535865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:09.536491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.536791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.537056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.537802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:09.538110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:09.539553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:09.553629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 510/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:09.554915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:09.640207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:09.673761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37efc1-d102-479a-8216-72df6205894f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:09.710238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:09.710960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.711252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.711534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] PATH_INFO: `/v3/users/537f8aaba1d94c2db6f0f2848e24dba5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.727297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] RBAC: Authorizing `identity:get_user(user_id=537f8aaba1d94c2db6f0f2848e24dba5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:09.728657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:09.735213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 540/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:09 2022] GET /identity/v3/users/537f8aaba1d94c2db6f0f2848e24dba5 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:09.736508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:09.841397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40541c1c-2256-49f2-9abb-0631dea0e11d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:09.877588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:09.878221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:09.878506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:09.878787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:09.933700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:09.957309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:09.958771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:10.065906 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 511/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 01:01:10.071685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:10.173645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188b9c-3a95-4982-bfb3-48febf171bee None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:10.203555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:10.211455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efff381-9adb-4014-a594-a8b0cf31042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:10.212101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efff381-9adb-4014-a594-a8b0cf31042b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.212396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efff381-9adb-4014-a594-a8b0cf31042b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.212672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efff381-9adb-4014-a594-a8b0cf31042b None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.213848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efff381-9adb-4014-a594-a8b0cf31042b None admin] RBAC: Authorizing `identity:get_project(project_id=2fce0f6cc54d4dc3ae355be36427b9a0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:10.215308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efff381-9adb-4014-a594-a8b0cf31042b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:10.217153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 541/1052] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:10 2022] GET /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:10.223188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:10.315052 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:10.338645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ebc64-4ce4-4a4d-9d72-eab0ba452ec8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:10.353806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb1a5b-ae59-4da9-b348-eb5cb3d6efe0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.354158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb1a5b-ae59-4da9-b348-eb5cb3d6efe0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.354382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb1a5b-ae59-4da9-b348-eb5cb3d6efe0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.355926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 542/1053] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:01:10.360901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.361198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.361482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.379403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:10.380297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:10.380428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.380681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.380953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.381507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:10.381755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0, user.id=537f8aaba1d94c2db6f0f2848e24dba5) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:10.382069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:10.383573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:10.389408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 512/1054] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:10 2022] GET /identity/v3/role_assignments?user.id=537f8aaba1d94c2db6f0f2848e24dba5&scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:10.472473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 543/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:10.495365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:10.613717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6b833-aa74-4105-83be-de87fb48d0dc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:10.652740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:10.653774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.654241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.654692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] PATH_INFO: `/v3/services/image` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.655525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:10.657076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:10.666642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:10.667706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 01:01:10.668547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:10.674404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 513/1056] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jul 28 01:01:10 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:10.676084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:10.780668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ce1b8-b984-40f6-90b3-b6026eb0cf51 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:10.809678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:10.820272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:10.820917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.821200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.821467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.822168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:10.822470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:10.823900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:10.834180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 544/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:10.836003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:10.848154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:10.952409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56c107-db99-4e5a-ac3f-194cd03ce899 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:10.988228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:10.988859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:10.989142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:10.989418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:10.990111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] RBAC: Adding query filter params (type=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:10.990416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:10.991923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb8f7-99e3-4b10-9117-d2342eaa7b6c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:11.001893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 514/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:11.003591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:11.113301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496ab40-4b7e-4d7b-ab80-5d32f40ed6c7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:11.150933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a117d2-704b-4203-9477-8c4f32d48c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:11.151539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a117d2-704b-4203-9477-8c4f32d48c76 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:11.151819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a117d2-704b-4203-9477-8c4f32d48c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:11.152103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a117d2-704b-4203-9477-8c4f32d48c76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:11.152901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a117d2-704b-4203-9477-8c4f32d48c76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:11.154468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a117d2-704b-4203-9477-8c4f32d48c76 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:11.181331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 545/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:01:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:11.204396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3709e-8210-4ea7-b43d-6588ab830a63 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:11.204586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3709e-8210-4ea7-b43d-6588ab830a63 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:11.204859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3709e-8210-4ea7-b43d-6588ab830a63 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:11.206314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 515/1060] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:11.212413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:11.212668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:11.212965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:11.232956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:11.234237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:11.284233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:11.319717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 546/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:11.343410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3709e-8210-4ea7-b43d-6588ab830a63 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:11.424760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:11.457426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3709e-8210-4ea7-b43d-6588ab830a63 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:11.496364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:11.497197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:11.497351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:11.497619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:11.510632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:11.515717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:11.516682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:11.517540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 516/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 01:01:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:11.523087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:11.638522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f4ee4-8505-4117-af47-6bc32e742fb7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:11.677412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:11.678105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:11.678325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:11.678591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:11.679494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:11.679805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:11.681176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:11.690124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 547/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:01:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:11.695985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:11.804404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cea21-ef4c-4c75-a8d4-454a42e53512 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:11.834256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:11.840985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:11.841635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:11.841915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:11.842188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:11.852840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:11.854280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:11.869475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:11.870518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:11.872326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:11.872326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:11.872326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:11.872326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:11.872326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 517/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:11.877075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:11.918134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:11.991343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60399b-31e3-4c58-8152-5fff236a0aef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.028345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.028983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.029265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.029539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.030243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:12.030557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.031969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.044502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 548/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:12.050299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.156730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286d31b-b3ce-4e73-8e42-5c9504cfe93c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.193320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.193977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.194264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.194542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.202594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.204087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-814e390e-2504-46b2-a083-d9e9244582da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:01:12.216618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:12.217707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:01:12.218980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:12.222696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 518/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:12.224200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.327144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:12.334562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83d64b-2e05-4a70-ab3f-d342d4695105 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.372050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.372688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.372966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.373238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.374037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:12.374344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.375821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.395293 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 549/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:12.399891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d447-be4e-41c8-929e-59d851458e47 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.400176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d447-be4e-41c8-929e-59d851458e47 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.400446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d447-be4e-41c8-929e-59d851458e47 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.400718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.401954 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 519/1068] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:01:12.406826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.407085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.407340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.425501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:12.426703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:12.480495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:12.509458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 520/1069] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:12.523267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4eb56-142e-436e-97a0-37c25ca20835 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.531185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.560041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.560694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.560981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.561261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.572882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.574349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:12.590624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:12.592615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:12.594653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:12.594653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:12.594653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:12.594653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:12.596469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 550/1070] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:12.598209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.609651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:12.667226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b900e68-cd59-4255-9d9d-49278f52c13e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.704517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.705213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.705478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.705748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.717910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:12.723128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:12.724082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:12.724917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 521/1071] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:12.731033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.732968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00023-9278-4764-ab83-9c8ba9b56192 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.768652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:12.773558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.774203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.774489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.774766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.775948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:12.775948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.777289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.791116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 551/1072] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:12.796797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.850558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837511cb-37e3-43f3-8755-49c4dba21c90 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.864423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:12.896885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.897501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.897781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.898110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.898872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:12.899170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.900624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.902582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9e6c0-1753-434e-9c73-74cc2981ee11 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:12.909636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 522/1073] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:12.915704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:12.944414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:12.945182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:12.945471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:12.945743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:12.953810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:12.955291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:01:12.967051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:12.968203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:12.969216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 552/1074] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:12.974551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:13.040920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:13.048902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46da0-c859-4265-9e0f-c5986ff07b1a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:13.096732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:13.097818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.098112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.098387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] PATH_INFO: `/v3/users/537f8aaba1d94c2db6f0f2848e24dba5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.099633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] RBAC: Authorizing `identity:get_user(user_id=537f8aaba1d94c2db6f0f2848e24dba5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:13.101112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:13.102770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 523/1075] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/users/537f8aaba1d94c2db6f0f2848e24dba5 => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:13.108679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:13.116113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd12c-3798-4822-92fe-3bd5fc9b6b4c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:13.143402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:13.154350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:13.155367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.155858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.156284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.157250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:13.157601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:13.159089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:13.175212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 553/1076] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:13.179061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:13.223725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c78562-c6b6-48db-a654-d53889573224 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:13.261082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:13.261730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.262054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.262284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.263497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] RBAC: Authorizing `identity:get_project(project_id=2fce0f6cc54d4dc3ae355be36427b9a0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:13.265156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:13.266783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 524/1077] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:13 2022] GET /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:13.272703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:13.301082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ba0b-22e9-44b3-9367-9ea7bcac044c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:13.305725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:13.336090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a53c4-490d-4bdd-85cb-cf1d0d333f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:13.336865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a53c4-490d-4bdd-85cb-cf1d0d333f8e None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.337116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a53c4-490d-4bdd-85cb-cf1d0d333f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.337515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a53c4-490d-4bdd-85cb-cf1d0d333f8e None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8f2f77d4f4c44890ac1ab88ff66100bd/roles/cdb572ff146e4cf286b3ab8fa28c6a36` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.339996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a53c4-490d-4bdd-85cb-cf1d0d333f8e None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=8f2f77d4f4c44890ac1ab88ff66100bd, role_id=cdb572ff146e4cf286b3ab8fa28c6a36)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:13.341441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a53c4-490d-4bdd-85cb-cf1d0d333f8e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:13.356690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 554/1078] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:01:13 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/8f2f77d4f4c44890ac1ab88ff66100bd/roles/cdb572ff146e4cf286b3ab8fa28c6a36 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:01:13.368410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad15b-a73b-4f4c-a1ad-0510cef954d5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.368689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad15b-a73b-4f4c-a1ad-0510cef954d5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.368963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad15b-a73b-4f4c-a1ad-0510cef954d5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.370486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 555/1079] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:13.376625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.377122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.377435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.389695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f794c-a53f-4563-83ec-ddb7df7e5b84 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:13.403184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:13.404484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:13.438145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:13.438885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.439194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.439465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0/users/537f8aaba1d94c2db6f0f2848e24dba5/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.441681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] RBAC: Authorizing `identity:create_grant(project_id=2fce0f6cc54d4dc3ae355be36427b9a0, user_id=537f8aaba1d94c2db6f0f2848e24dba5, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:13.443297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:13.459218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:13.466051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 525/1080] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:01:13 2022] PUT /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0/users/537f8aaba1d94c2db6f0f2848e24dba5/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:01:13.490875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 556/1081] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:13.514950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:13.638529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:13.655125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feab0b-8983-4a5f-8b8e-d36adea62a6c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:13.692484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:13.693138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:13.693429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:13.693705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:13.771861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:13.773287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:13.872433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:13.884717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 526/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:13.891161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:13.997685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e2c2f-ef94-4f9f-aeb7-d5e03e12322a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:14.034010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:14.034586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:14.034899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:14.035179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:14.045853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:14.047339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f1e15f22-a274-4185-9143-de30fc3a1844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:14.063016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:14.063983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:14.064861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:14.064861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:14.064861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:14.064861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:14.064861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 557/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:14.070327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:14.184812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf19a1-6fcb-46e7-9f1b-13ea5d66d326 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:14.221080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:14.221718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:14.222002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:14.222281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:14.223018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:14.223329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:14.224719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8215c-2f8e-4685-8d68-fddb20a169c2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:14.236881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 527/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:15.393336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3318bb7-f923-4d1f-ace9-7d562113d60b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.393948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3318bb7-f923-4d1f-ace9-7d562113d60b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.393948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3318bb7-f923-4d1f-ace9-7d562113d60b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.395333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 558/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:15.401427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.401948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.402060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.421761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:15.422997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:15.479154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a58b-04fc-4199-af1a-92c6d2f5d103 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.479441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a58b-04fc-4199-af1a-92c6d2f5d103 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.479718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a58b-04fc-4199-af1a-92c6d2f5d103 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.481033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:15.481154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 559/1086] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:01:15.486252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.486529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.486801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.505411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:15.506269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 528/1087] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:15.506592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:15.529603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:15.533329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:15.580295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 560/1088] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:15.603165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:15.637665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379dea6-1729-4430-8453-a57ef9f87fc2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:15.676853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:15.677510 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.677789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.678063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.688824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:15.693694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:15.694707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:15.701003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 529/1089] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jul 28 01:01:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:15.701734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:15.710934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a722-ae9c-456a-8479-d7b30f945c7a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:15.746596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:15.747272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.747551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.747824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.748635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:15.759556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:15.764572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:15.765573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:15.766433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 561/1090] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:15.772112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:15.793113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:15.810562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae14aac-1fcb-408a-8d76-a0ef3568d203 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:15.845853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:15.846493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.846782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.847092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.847825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:15.848132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:15.849529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:15.858305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 530/1091] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:01:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:15.864141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:15.878611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320eeee-822e-44bb-b5c2-6166b9d08530 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:15.914324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:15.915004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:15.915289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:15.915565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:15.916296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:15.916603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:15.917978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:15.926356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 562/1092] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:15.933331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:15.971677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327b132-162c-4b1f-b649-6264e53569a5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:15.974150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.008275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.008929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.009211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.009490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.016791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.020534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.022010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.036868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e215c0-0de7-4bcd-a9b7-2d0d83e82381 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:16.037117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:16.038158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:16.039383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:16.039383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:16.039383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:16.039383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:16.039383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 531/1093] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:01:16.044101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.072537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.073193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.073477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.073753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] PATH_INFO: `/v3/users/537f8aaba1d94c2db6f0f2848e24dba5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.074963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] RBAC: Authorizing `identity:get_user(user_id=537f8aaba1d94c2db6f0f2848e24dba5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.076398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.078058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 563/1094] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:15 2022] GET /identity/v3/users/537f8aaba1d94c2db6f0f2848e24dba5 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:16.083818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.149872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302fe0-848e-4169-8333-e115f0f83df2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.184053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d659cf2-8438-4b3b-a18c-d60be41e028b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.187645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.188283 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.188567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.188843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.189550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:16.189856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.191266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.207613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 532/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:16.208726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.220486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.221073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.221361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.221639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.228607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.240083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.291973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.293432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.312503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b033f-103e-40fe-b51a-fda29425f8b5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.348371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.348790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.349085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.349361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.357181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.358626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:01:16.370119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:16.371190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:16.372256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 533/1096] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:16.377207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.397833 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 564/1097] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 01:01:16.403863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.456366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.465322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.495888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1df8b-4df4-49ff-a8f5-f955cf22dcd8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.516777 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce62d47-9ee7-44d8-8704-a78ca884995b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.531663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.532471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.532750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.533023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.533928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:16.534229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.535748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.553239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 534/1098] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:16.554483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.555188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.555468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.555741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.556947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] RBAC: Authorizing `identity:get_project(project_id=2fce0f6cc54d4dc3ae355be36427b9a0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.558394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.559048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.560286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 565/1099] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:16.566114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:16.672189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf242c-c26b-4f20-b92b-a4a38b1aa16b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.678312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23effee-9967-4547-ae77-4526e071eb81 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:16.695705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.709016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.709576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.709862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.710136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.712896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:16.715111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:16.715931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:16.716224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:16.716502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:16.717311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0, user.id=537f8aaba1d94c2db6f0f2848e24dba5) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:16.717629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.719111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.725671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 566/1100] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/role_assignments?user.id=537f8aaba1d94c2db6f0f2848e24dba5&scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:16.789077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:16.790562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:16.916163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 535/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:16.923075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:17.034243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c5d98-42c3-410c-a3cd-d715e31fa822 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:17.075753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:17.076472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:17.076787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:17.077068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:17.088140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:17.089600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:17.104477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:17.105432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:17.106318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:01:17.106318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:17.106318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:01:17.106318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:17.106318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 567/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:17.111614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:17.148227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:17.227726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695daa6-761a-483c-be18-75de54fd145f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:17.267071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:17.267727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:17.268006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:17.268280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:17.268999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:17.269308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:17.270728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:17.283855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 536/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:17.289917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:17.324084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:17.405884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9a820-a2b8-4a75-9f39-10f890061b3d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:17.442432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:17.443122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:17.443410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:17.443684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:17.453506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:17.455382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:01:17.473801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:17.474805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:01:17.475761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:17.479190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 568/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:17.483597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:17.583733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:17.592004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80033f3a-6b4f-472c-a1b5-e3a51e0b0964 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:17.628603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:17.629238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:17.629529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:17.630170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:17.630945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:17.631267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:17.632773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7511e-3306-4d69-b973-98a82c1b7b7a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:17.645398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 537/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:01:17.651944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:17.750205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:17.760715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae8090-f355-4dfa-adcd-8007ad258fca None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:17.797492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:17.798133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:17.798421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:17.798674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:17.799553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] RBAC: Adding query filter params (role.id=cdb572ff146e4cf286b3ab8fa28c6a36, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=8f2f77d4f4c44890ac1ab88ff66100bd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:17.799878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:17.801332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce53d-a2f6-4397-bcb2-44b4b1f90839 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:17.808334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 569/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:17 2022] GET /identity/v3/role_assignments?user.id=8f2f77d4f4c44890ac1ab88ff66100bd&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=cdb572ff146e4cf286b3ab8fa28c6a36 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:18.797091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024f15-f4c0-489c-9ca8-5f2aeb4e6fc6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:18.797648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024f15-f4c0-489c-9ca8-5f2aeb4e6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:18.797648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024f15-f4c0-489c-9ca8-5f2aeb4e6fc6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:18.799170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 538/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:18.804960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:18.805386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:18.805518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:18.824939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:18.826161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:18.898516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 570/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:18.923392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66024f15-f4c0-489c-9ca8-5f2aeb4e6fc6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:19.030669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66024f15-f4c0-489c-9ca8-5f2aeb4e6fc6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:19.066728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcfdf1-d730-4ace-a453-85cea4481c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:19.067323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfdf1-d730-4ace-a453-85cea4481c41 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:19.067607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfdf1-d730-4ace-a453-85cea4481c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:19.067886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfdf1-d730-4ace-a453-85cea4481c41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:19.069168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcfdf1-d730-4ace-a453-85cea4481c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:19.070677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcfdf1-d730-4ace-a453-85cea4481c41 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:19.072325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 539/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:19.078607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:19.167553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:19.182756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4f6e1-bdfd-429b-801d-d3b1693a6806 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:19.216829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71ea2-3375-44f3-83e2-9c07828a5d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:19.217459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71ea2-3375-44f3-83e2-9c07828a5d0e None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:19.217738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71ea2-3375-44f3-83e2-9c07828a5d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:19.218007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71ea2-3375-44f3-83e2-9c07828a5d0e None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:19.218696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d71ea2-3375-44f3-83e2-9c07828a5d0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:19.220173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d71ea2-3375-44f3-83e2-9c07828a5d0e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:19.253062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 571/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:01:19 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:21.277037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9283-5753-4879-853b-997d2ff1af54 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:21.277612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9283-5753-4879-853b-997d2ff1af54 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:21.277612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9283-5753-4879-853b-997d2ff1af54 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:21.279038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 540/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:21.284341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:21.284790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:21.284883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:21.304711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:21.305930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:21.380494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 572/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:21.405160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9283-5753-4879-853b-997d2ff1af54 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:21.430128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:21.510534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9283-5753-4879-853b-997d2ff1af54 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:21.546106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:21.546683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:21.547008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:21.547287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] PATH_INFO: `/v3/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:21.548782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] RBAC: Authorizing `identity:get_role(role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:21.550174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:21.551561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 541/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:21 2022] GET /identity/v3/roles/8fc2a014f1e24e08a63b265436f41eec => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:21.561621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:21.639660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:21.673171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cbb8-f630-4a97-ac3c-dd4151c76d3f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:21.712500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:21.713154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:21.713436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:21.713719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] PATH_INFO: `/v3/users/76cec544bd9946ef838933f62f68ec06` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:21.727247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=76cec544bd9946ef838933f62f68ec06)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:21.728691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:21.730355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 573/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:21 2022] GET /identity/v3/users/76cec544bd9946ef838933f62f68ec06 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:21.737171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:21.832996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:21.844141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06164b00-61ce-4778-8041-091a686a62b3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:21.878067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:21.878692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:21.879015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:21.879288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:21.948511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:21.949964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:22.040685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:22.052455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 542/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:22.058636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:22.163231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4458d8d-b4a8-447a-a793-d8fa621f8ec0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:22.194754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:22.201602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5a4b3-a5da-48aa-a8d0-3feb929d421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:22.202260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5a4b3-a5da-48aa-a8d0-3feb929d421b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:22.202537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5a4b3-a5da-48aa-a8d0-3feb929d421b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:22.202809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5a4b3-a5da-48aa-a8d0-3feb929d421b None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:22.204147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5a4b3-a5da-48aa-a8d0-3feb929d421b None admin] RBAC: Authorizing `identity:get_project(project_id=2fce0f6cc54d4dc3ae355be36427b9a0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:22.205589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5a4b3-a5da-48aa-a8d0-3feb929d421b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:22.207324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 574/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:22 2022] GET /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:22.213886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:22.319185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaee2b-bce9-456b-8ac2-2b7ac5e235f1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:22.355195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:22.355816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:22.356102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:22.356379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:22.357158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] RBAC: Adding query filter params (role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0, user.id=76cec544bd9946ef838933f62f68ec06) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:22.357481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:22.358941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ad4f-acf5-4879-b9a5-8139b8cb881e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:22.364404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 543/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:22 2022] GET /identity/v3/role_assignments?user.id=76cec544bd9946ef838933f62f68ec06&scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:24.336061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef80949-7107-42d8-883a-04873f2dda0f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:24.336630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef80949-7107-42d8-883a-04873f2dda0f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:24.336630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef80949-7107-42d8-883a-04873f2dda0f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:24.338027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 575/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:24.343526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:24.343950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:24.344213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:24.369919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:24.371170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:24.429036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:24.448929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 544/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:24.471581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef80949-7107-42d8-883a-04873f2dda0f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:24.579678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef80949-7107-42d8-883a-04873f2dda0f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:24.615180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:24.615745 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:24.616032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:24.616315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] PATH_INFO: `/v3/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:24.617876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] RBAC: Authorizing `identity:get_role(role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:24.619307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:24.620712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 576/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:24 2022] GET /identity/v3/roles/8fc2a014f1e24e08a63b265436f41eec => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:24.626999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:24.737177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ea4f0-c346-4ffc-b717-71655de86dc4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:24.771291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:24.771941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:24.772227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:24.772509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] PATH_INFO: `/v3/users/76cec544bd9946ef838933f62f68ec06` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:24.773757 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] RBAC: Authorizing `identity:get_user(user_id=76cec544bd9946ef838933f62f68ec06)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:24.775229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:24.777044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 545/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:24 2022] GET /identity/v3/users/76cec544bd9946ef838933f62f68ec06 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:24.783033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:24.851351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:24.888012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00e8b-fe79-4774-a9cf-0eec82390c67 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:24.923218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210db10-2b67-40ac-91f8-f4553355c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:24.923874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210db10-2b67-40ac-91f8-f4553355c74e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:24.924162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210db10-2b67-40ac-91f8-f4553355c74e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:24.924440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210db10-2b67-40ac-91f8-f4553355c74e None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:24.925645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210db10-2b67-40ac-91f8-f4553355c74e None admin] RBAC: Authorizing `identity:get_project(project_id=2fce0f6cc54d4dc3ae355be36427b9a0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:24.927117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210db10-2b67-40ac-91f8-f4553355c74e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:24.928793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 577/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:24 2022] GET /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:24.935175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:24.990937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:25.040766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4972e-2090-4806-aebe-e6b10bf7c011 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:25.076432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd245-78a0-45f0-9774-1c74ae400cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:25.077136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd245-78a0-45f0-9774-1c74ae400cf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:25.077419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd245-78a0-45f0-9774-1c74ae400cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:25.077697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd245-78a0-45f0-9774-1c74ae400cf7 None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0/users/76cec544bd9946ef838933f62f68ec06/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:25.079996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd245-78a0-45f0-9774-1c74ae400cf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fce0f6cc54d4dc3ae355be36427b9a0, user_id=76cec544bd9946ef838933f62f68ec06, role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:25.081446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd245-78a0-45f0-9774-1c74ae400cf7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:25.095997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 546/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:01:24 2022] PUT /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0/users/76cec544bd9946ef838933f62f68ec06/roles/8fc2a014f1e24e08a63b265436f41eec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:01:27.054111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b577-5c07-4f4c-b944-7422ac3b35bf None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:27.054681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b577-5c07-4f4c-b944-7422ac3b35bf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:27.054681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b577-5c07-4f4c-b944-7422ac3b35bf None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:27.056154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 578/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:27.061735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:27.062137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:27.062296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:27.081928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:27.083162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:27.152721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 547/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:27.176111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b577-5c07-4f4c-b944-7422ac3b35bf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:27.283540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b577-5c07-4f4c-b944-7422ac3b35bf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:27.318875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:27.319459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:27.319773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:27.320088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] PATH_INFO: `/v3/roles/8fc2a014f1e24e08a63b265436f41eec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:27.321602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] RBAC: Authorizing `identity:get_role(role_id=8fc2a014f1e24e08a63b265436f41eec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:27.323006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:27.324378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 579/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:27 2022] GET /identity/v3/roles/8fc2a014f1e24e08a63b265436f41eec => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:27.330624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:27.375397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:27.434396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92765ec-5ad2-424b-bff6-c4c7a45dd519 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:27.472925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:27.473589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:27.473867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:27.474142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] PATH_INFO: `/v3/users/76cec544bd9946ef838933f62f68ec06` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:27.475362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] RBAC: Authorizing `identity:get_user(user_id=76cec544bd9946ef838933f62f68ec06)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:27.476835 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:27.478470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 548/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:27 2022] GET /identity/v3/users/76cec544bd9946ef838933f62f68ec06 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:27.484417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:27.487341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:27.588708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cc637-9266-4ad3-9318-6324c40709e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:27.630708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:27.631378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:27.631665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:27.631937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:27.703700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:27.705160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:27.707059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:27.809169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 580/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:27.815456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:27.918533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131dd2e-3929-431c-a1bc-bae228202052 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:27.928900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:27.954468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92343a16-faaf-4376-8aaf-4d13175449e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:27.955149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92343a16-faaf-4376-8aaf-4d13175449e7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:27.955436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92343a16-faaf-4376-8aaf-4d13175449e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:27.955711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92343a16-faaf-4376-8aaf-4d13175449e7 None admin] PATH_INFO: `/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:27.956910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92343a16-faaf-4376-8aaf-4d13175449e7 None admin] RBAC: Authorizing `identity:get_project(project_id=2fce0f6cc54d4dc3ae355be36427b9a0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:27.958353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92343a16-faaf-4376-8aaf-4d13175449e7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:27.960055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 549/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:27 2022] GET /identity/v3/projects/2fce0f6cc54d4dc3ae355be36427b9a0 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:27.966577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:28.069838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0407415-54a8-4402-ad36-91cf551b32ad None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:28.071395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:28.108562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:28.109203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:28.109490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:28.109780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:28.110571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] RBAC: Adding query filter params (role.id=8fc2a014f1e24e08a63b265436f41eec, scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0, user.id=76cec544bd9946ef838933f62f68ec06) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:28.110914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:28.112368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f387b4-3785-44fa-8191-79a9dbb33be3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:28.118558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 581/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:27 2022] GET /identity/v3/role_assignments?user.id=76cec544bd9946ef838933f62f68ec06&scope.project.id=2fce0f6cc54d4dc3ae355be36427b9a0&role.id=8fc2a014f1e24e08a63b265436f41eec => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:30.141740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aed48-48c2-4232-b448-e70271fedb32 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:30.142341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aed48-48c2-4232-b448-e70271fedb32 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:30.142341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aed48-48c2-4232-b448-e70271fedb32 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:30.143681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 550/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:30.149208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:30.149633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:30.149743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:30.169406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:30.170614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:30.240257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 582/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:30.264050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aed48-48c2-4232-b448-e70271fedb32 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:30.374813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aed48-48c2-4232-b448-e70271fedb32 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:30.410088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df8d7c-ed30-4c55-bcf6-3db3efe6b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:30.410726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df8d7c-ed30-4c55-bcf6-3db3efe6b715 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:30.411016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df8d7c-ed30-4c55-bcf6-3db3efe6b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:30.411290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df8d7c-ed30-4c55-bcf6-3db3efe6b715 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:30.412514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df8d7c-ed30-4c55-bcf6-3db3efe6b715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:30.413958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df8d7c-ed30-4c55-bcf6-3db3efe6b715 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:30.415597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 551/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:30.421973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:30.518329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:30.524388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314ca37-da55-4c57-9a34-cb0e18fa4caf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:30.559080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746d19d-e886-404d-b4f5-a757408a75c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:30.559706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746d19d-e886-404d-b4f5-a757408a75c9 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:30.559984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746d19d-e886-404d-b4f5-a757408a75c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:30.560257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746d19d-e886-404d-b4f5-a757408a75c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:30.560938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746d19d-e886-404d-b4f5-a757408a75c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:30.562359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746d19d-e886-404d-b4f5-a757408a75c9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:30.591405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 583/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:01:30 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:32.621182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec5519-0bd7-41d3-8531-7d30249dad4c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:32.621722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec5519-0bd7-41d3-8531-7d30249dad4c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:32.621722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec5519-0bd7-41d3-8531-7d30249dad4c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:32.623169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 552/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:32.628864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:32.629361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:32.629459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:32.649021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:32.650231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:32.719794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 584/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:32.744494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec5519-0bd7-41d3-8531-7d30249dad4c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:32.800327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:32.855558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec5519-0bd7-41d3-8531-7d30249dad4c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:32.891223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3232c2d-e715-4c0c-ae86-8600c59c5a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:32.891867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232c2d-e715-4c0c-ae86-8600c59c5a6b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:32.892155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232c2d-e715-4c0c-ae86-8600c59c5a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:32.892438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232c2d-e715-4c0c-ae86-8600c59c5a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:32.893657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3232c2d-e715-4c0c-ae86-8600c59c5a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:32.895132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3232c2d-e715-4c0c-ae86-8600c59c5a6b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:32.896735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 553/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:32.902871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:32.983429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:33.016066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33807b-641b-430a-b78b-f0efe2a52f40 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:33.049377 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3409-6345-4fc5-bfe4-f822acd30891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:33.050004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3409-6345-4fc5-bfe4-f822acd30891 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:33.050296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3409-6345-4fc5-bfe4-f822acd30891 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:33.050572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3409-6345-4fc5-bfe4-f822acd30891 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:33.051286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d3409-6345-4fc5-bfe4-f822acd30891 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:33.052729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d3409-6345-4fc5-bfe4-f822acd30891 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:33.074627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 585/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:01:32 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:35.101444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a4f5-c99c-4d84-a4fb-36b0dfb1f46f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:35.102083 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a4f5-c99c-4d84-a4fb-36b0dfb1f46f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:35.102083 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a4f5-c99c-4d84-a4fb-36b0dfb1f46f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:35.103374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 554/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:35.108844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:35.109281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:35.109413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:35.129058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:35.130648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:35.204879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 586/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:35.228618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a4f5-c99c-4d84-a4fb-36b0dfb1f46f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:35.335786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a4f5-c99c-4d84-a4fb-36b0dfb1f46f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:35.350866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:35.372901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:35.373473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:35.373748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:35.373997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:35.384787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:35.389656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:35.390710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:35.391711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 555/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:35.397222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:35.464672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:35.503978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a867b8-6173-4e53-a9c6-ff380ee3dca5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:35.540519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:35.541159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:35.541442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:35.541721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:35.542449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:35.542759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:35.544162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:35.552908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 587/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:35.559246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:35.668345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb88979-1913-4340-beac-9193eab89b2d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:35.704441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:35.705093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:35.705375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:35.705653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] PATH_INFO: `/v3/users/e0ead44b3dc1495c9f0683ae8104bbeb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:35.720041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ead44b3dc1495c9f0683ae8104bbeb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:35.721512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:35.723221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 556/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:35 2022] GET /identity/v3/users/e0ead44b3dc1495c9f0683ae8104bbeb => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:35.729391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:35.842585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bdf8e-c40a-4b30-a4e6-f97c4195de2e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:35.869277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:35.878595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:35.879260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:35.879543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:35.880183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:35.953165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:35.954532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:36.078576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 588/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:36.084942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:36.121787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:36.196602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53376994-dd67-4fa4-a28f-bce8958a9217 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:36.232079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11810fc6-d4d4-402c-afcd-b354aea09c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:36.232625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11810fc6-d4d4-402c-afcd-b354aea09c9e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:36.232902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11810fc6-d4d4-402c-afcd-b354aea09c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:36.233177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11810fc6-d4d4-402c-afcd-b354aea09c9e None admin] PATH_INFO: `/v3/projects/4c10e18cca6b4cebb90d75e35f253424` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:36.234378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11810fc6-d4d4-402c-afcd-b354aea09c9e None admin] RBAC: Authorizing `identity:get_project(project_id=4c10e18cca6b4cebb90d75e35f253424)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:36.235833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11810fc6-d4d4-402c-afcd-b354aea09c9e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:36.237505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 557/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:36 2022] GET /identity/v3/projects/4c10e18cca6b4cebb90d75e35f253424 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:36.243880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:36.279838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:36.357153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46100fbe-1d4e-4a71-a93c-d304ae15c1c9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:36.394931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:36.395563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:36.395847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:36.396132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:36.396931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=4c10e18cca6b4cebb90d75e35f253424, user.id=e0ead44b3dc1495c9f0683ae8104bbeb) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:36.397245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:36.398769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13eb78-9c0a-4046-8160-37e20c271246 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:36.404361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 589/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:36 2022] GET /identity/v3/role_assignments?user.id=e0ead44b3dc1495c9f0683ae8104bbeb&scope.project.id=4c10e18cca6b4cebb90d75e35f253424&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:38.357095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f88bc-281e-45ae-a2c8-73cf45dad873 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:38.357663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f88bc-281e-45ae-a2c8-73cf45dad873 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:38.357663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f88bc-281e-45ae-a2c8-73cf45dad873 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:38.359021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 558/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:38.364305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:38.364737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:38.364828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:38.383687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:38.384868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:38.453123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 590/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:38.474990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f88bc-281e-45ae-a2c8-73cf45dad873 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:38.577642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:38.582060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f88bc-281e-45ae-a2c8-73cf45dad873 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:38.616228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:38.616850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:38.617137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:38.617415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:38.628061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:38.633779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:38.634788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:38.635846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 559/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:38.641455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:38.663155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:38.748030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34973adb-3cdd-47dd-8df3-acf625d16294 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:38.782334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:38.782996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:38.783286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:38.783556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:38.784253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:38.784557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:38.785917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:38.793954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 591/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:38.800123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:38.904434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8284f9-200b-409d-a148-8c6e9249f56d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:38.939147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:38.939787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:38.940066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:38.940320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] PATH_INFO: `/v3/users/e0ead44b3dc1495c9f0683ae8104bbeb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:38.941494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] RBAC: Authorizing `identity:get_user(user_id=e0ead44b3dc1495c9f0683ae8104bbeb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:38.942941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:38.944605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 560/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:38 2022] GET /identity/v3/users/e0ead44b3dc1495c9f0683ae8104bbeb => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:38.950927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:39.047662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:39.068781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921943b-76e3-4cc0-815f-c497d5517854 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:39.109040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4a2fe-6ef0-4782-bed3-dbd3c2fe97ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:39.109467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4a2fe-6ef0-4782-bed3-dbd3c2fe97ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:39.109743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4a2fe-6ef0-4782-bed3-dbd3c2fe97ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:39.110017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4a2fe-6ef0-4782-bed3-dbd3c2fe97ac None admin] PATH_INFO: `/v3/projects/4c10e18cca6b4cebb90d75e35f253424` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:39.111239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4a2fe-6ef0-4782-bed3-dbd3c2fe97ac None admin] RBAC: Authorizing `identity:get_project(project_id=4c10e18cca6b4cebb90d75e35f253424)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:39.112672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4a2fe-6ef0-4782-bed3-dbd3c2fe97ac None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:39.114394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 592/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:38 2022] GET /identity/v3/projects/4c10e18cca6b4cebb90d75e35f253424 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:39.121226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:39.154093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:39.227270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc91eb-2323-4bd2-802c-4e7a70e4204f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:39.261150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62de02-0724-438b-ae56-58440774e984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:39.261825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62de02-0724-438b-ae56-58440774e984 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:39.262115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62de02-0724-438b-ae56-58440774e984 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:39.262396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62de02-0724-438b-ae56-58440774e984 None admin] PATH_INFO: `/v3/projects/4c10e18cca6b4cebb90d75e35f253424/users/e0ead44b3dc1495c9f0683ae8104bbeb/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:39.264490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62de02-0724-438b-ae56-58440774e984 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c10e18cca6b4cebb90d75e35f253424, user_id=e0ead44b3dc1495c9f0683ae8104bbeb, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:39.265946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62de02-0724-438b-ae56-58440774e984 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:39.280768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 561/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:01:39 2022] PUT /identity/v3/projects/4c10e18cca6b4cebb90d75e35f253424/users/e0ead44b3dc1495c9f0683ae8104bbeb/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:01:41.275297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3aa2b-67fe-41ed-89ad-111fcb2c4abc None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:41.275866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3aa2b-67fe-41ed-89ad-111fcb2c4abc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:41.275866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3aa2b-67fe-41ed-89ad-111fcb2c4abc None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:41.277200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 593/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:41.282576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:41.283041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:41.283153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:41.303098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:41.304296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:41.376100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 562/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:41.400317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3aa2b-67fe-41ed-89ad-111fcb2c4abc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:41.509330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3aa2b-67fe-41ed-89ad-111fcb2c4abc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:41.545838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:41.546404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:41.546685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:41.546993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:41.557716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:41.562679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:41.563617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:41.564456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 594/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:41.570530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:41.590702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:41.679574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84414695-9ffd-473d-8137-b502c01768b1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:41.714386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:41.715038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:41.715315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:41.715590 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:41.716309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:41.716614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:41.717976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:41.726420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 563/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:41.732727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:41.746016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:41.843185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753c1b3-b7fc-4b49-a4ce-2d729dc7b9e7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:41.881784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:41.882445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:41.882731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:41.883043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] PATH_INFO: `/v3/users/e0ead44b3dc1495c9f0683ae8104bbeb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:41.884295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] RBAC: Authorizing `identity:get_user(user_id=e0ead44b3dc1495c9f0683ae8104bbeb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:41.885750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:41.887450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 595/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:41 2022] GET /identity/v3/users/e0ead44b3dc1495c9f0683ae8104bbeb => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:41.893565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:41.976917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:41.998490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719024-b489-4f66-8477-a12f15a9456b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:42.041204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:42.041840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:42.042124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:42.042402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:42.115770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:42.117172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:42.198082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:42.223522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 564/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:42.229781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:42.310238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:42.333067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f034247-948f-45d0-bb8a-295fc21787f9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:42.370793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914df0c9-4693-4742-833f-f8d47202027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:42.371398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914df0c9-4693-4742-833f-f8d47202027d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:42.371688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914df0c9-4693-4742-833f-f8d47202027d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:42.371962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914df0c9-4693-4742-833f-f8d47202027d None admin] PATH_INFO: `/v3/projects/4c10e18cca6b4cebb90d75e35f253424` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:42.373179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914df0c9-4693-4742-833f-f8d47202027d None admin] RBAC: Authorizing `identity:get_project(project_id=4c10e18cca6b4cebb90d75e35f253424)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:42.374618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914df0c9-4693-4742-833f-f8d47202027d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:42.376329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 596/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:42 2022] GET /identity/v3/projects/4c10e18cca6b4cebb90d75e35f253424 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:42.383337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:42.488091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9eac-6e41-4096-832d-426ec55b07c7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:42.524502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:42.525139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:42.525425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:42.525705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:42.526488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=4c10e18cca6b4cebb90d75e35f253424, user.id=e0ead44b3dc1495c9f0683ae8104bbeb) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:42.526803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:42.528285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d2af-b918-47de-b787-bbe74e82bea2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:42.534429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 565/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:42 2022] GET /identity/v3/role_assignments?user.id=e0ead44b3dc1495c9f0683ae8104bbeb&scope.project.id=4c10e18cca6b4cebb90d75e35f253424&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:44.541107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06cc36-f11e-4623-9901-cb3de16746ee None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:44.541922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06cc36-f11e-4623-9901-cb3de16746ee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:44.541922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06cc36-f11e-4623-9901-cb3de16746ee None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:44.544288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 597/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:44.550814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:44.551309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:44.551405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:44.570118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:44.571359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:44.614189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:44.649566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 566/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:01:44.671983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06cc36-f11e-4623-9901-cb3de16746ee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:44.779550 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06cc36-f11e-4623-9901-cb3de16746ee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:44.816504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:44.817130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:44.817412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:44.817685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:44.834007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:44.888837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:44.890314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:44.999058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 598/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:45.005975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:45.113114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee48f0e-4a46-4bbb-b6aa-95210f6bbc4d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:45.148710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:45.149352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:45.149639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:45.149918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:45.156815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:45.158275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d8544f7a-bcf4-41cc-bce3-3425b7fb1cfb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:45.170168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:45.171426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:01:45.172628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 567/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:01:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:45.177615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:45.226348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:45.284725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fba40c-a67f-48e0-8ddd-2654b4485479 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:45.320693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:45.321322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:45.321605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:45.321877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:45.322579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:45.322872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:45.324276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd4f8a-0ec2-430b-a2c1-a7e069c3aa55 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:45.331615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 599/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jul 28 01:01:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:47.325372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aced03-1808-4df0-b5a5-67de9d8263a3 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:47.325970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aced03-1808-4df0-b5a5-67de9d8263a3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:47.325970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aced03-1808-4df0-b5a5-67de9d8263a3 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:47.327358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 568/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:47.333713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15089d3-eb6e-490e-a133-5b1d5c20a457 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:47.334450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15089d3-eb6e-490e-a133-5b1d5c20a457 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:47.334642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15089d3-eb6e-490e-a133-5b1d5c20a457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:47.359756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d15089d3-eb6e-490e-a133-5b1d5c20a457 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:47.361019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d15089d3-eb6e-490e-a133-5b1d5c20a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:47.430685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 600/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:47.459214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aced03-1808-4df0-b5a5-67de9d8263a3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:47.507241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:47.581568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aced03-1808-4df0-b5a5-67de9d8263a3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:47.622971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a4e2-7b01-484d-a4b7-caa5c1348e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:47.625153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a4e2-7b01-484d-a4b7-caa5c1348e7f None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:47.625153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a4e2-7b01-484d-a4b7-caa5c1348e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:47.625153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a4e2-7b01-484d-a4b7-caa5c1348e7f None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:47.625153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a4e2-7b01-484d-a4b7-caa5c1348e7f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:47.626222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a4e2-7b01-484d-a4b7-caa5c1348e7f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:47.644766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 569/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jul 28 01:01:47 2022] POST /identity/v3/domains => generated 267 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:49.668643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211af56-4f49-49d3-9c01-ea7e766f34fc None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:49.669234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211af56-4f49-49d3-9c01-ea7e766f34fc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:49.669234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211af56-4f49-49d3-9c01-ea7e766f34fc None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:49.670562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 601/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:49.675882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:49.676349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:49.676440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:49.697399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:49.698625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:49.770179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 570/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:49.793116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211af56-4f49-49d3-9c01-ea7e766f34fc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:49.848472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:49.913957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211af56-4f49-49d3-9c01-ea7e766f34fc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:49.955000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac91cd-9960-4d07-a80b-240cd49859a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:49.955642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac91cd-9960-4d07-a80b-240cd49859a9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:49.955928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac91cd-9960-4d07-a80b-240cd49859a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:49.956212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac91cd-9960-4d07-a80b-240cd49859a9 None admin] PATH_INFO: `/v3/domains/b955f008e39c40efb8f58f2f06f33b90` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:49.967292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac91cd-9960-4d07-a80b-240cd49859a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b955f008e39c40efb8f58f2f06f33b90)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:49.968734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac91cd-9960-4d07-a80b-240cd49859a9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:49.970380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 602/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 01:01:49 2022] GET /identity/v3/domains/b955f008e39c40efb8f58f2f06f33b90 => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:49.976735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:50.003718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:50.082087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470cd99-c2cd-4750-9897-7e9a653a6adb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:50.118314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723c8d8-ac8d-4708-b543-4a6c78c10b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:50.118971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723c8d8-ac8d-4708-b543-4a6c78c10b13 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:50.119258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723c8d8-ac8d-4708-b543-4a6c78c10b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:50.119537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723c8d8-ac8d-4708-b543-4a6c78c10b13 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:50.120227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723c8d8-ac8d-4708-b543-4a6c78c10b13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:50.121675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723c8d8-ac8d-4708-b543-4a6c78c10b13 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:50.150183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 571/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:01:49 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:52.147491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf26257-f08e-4091-9a7f-1062f519bc3a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:52.148098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf26257-f08e-4091-9a7f-1062f519bc3a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:52.148098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf26257-f08e-4091-9a7f-1062f519bc3a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:52.149423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 603/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:52.155480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:52.156082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:52.156082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:52.177993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:52.179370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:52.253204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 572/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:52.283104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf26257-f08e-4091-9a7f-1062f519bc3a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:52.387022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:52.391158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf26257-f08e-4091-9a7f-1062f519bc3a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:52.426136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c47301-99d0-4b70-a9b0-000e00e8434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:52.426712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47301-99d0-4b70-a9b0-000e00e8434e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:52.427025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47301-99d0-4b70-a9b0-000e00e8434e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:52.427305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c47301-99d0-4b70-a9b0-000e00e8434e None admin] PATH_INFO: `/v3/domains/b955f008e39c40efb8f58f2f06f33b90` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:52.428563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c47301-99d0-4b70-a9b0-000e00e8434e None admin] RBAC: Authorizing `identity:get_domain(domain_id=b955f008e39c40efb8f58f2f06f33b90)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:52.430015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c47301-99d0-4b70-a9b0-000e00e8434e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:52.431662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 604/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jul 28 01:01:52 2022] GET /identity/v3/domains/b955f008e39c40efb8f58f2f06f33b90 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:52.437815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:52.446484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:52.542592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8449a-94a6-4228-b6d6-07947b7af38c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:52.586403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75bdf1-92e2-4865-b78f-50acb2f07770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:52.587055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75bdf1-92e2-4865-b78f-50acb2f07770 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:52.587342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75bdf1-92e2-4865-b78f-50acb2f07770 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:52.587623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75bdf1-92e2-4865-b78f-50acb2f07770 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:52.588314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75bdf1-92e2-4865-b78f-50acb2f07770 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:52.589767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75bdf1-92e2-4865-b78f-50acb2f07770 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:52.613240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 573/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:01:52 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:01:54.644681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e630ce-6a58-4adb-80e4-63d3f61219f9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:54.645281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e630ce-6a58-4adb-80e4-63d3f61219f9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:54.645281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e630ce-6a58-4adb-80e4-63d3f61219f9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:54.646601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 605/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:54.653549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:54.654091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:54.654183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:54.674131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:54.675451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:54.736979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:54.751259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 574/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:54.774582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e630ce-6a58-4adb-80e4-63d3f61219f9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:54.892478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e630ce-6a58-4adb-80e4-63d3f61219f9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:54.930111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104308-b660-47a1-b884-27b37f91115a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:54.930755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104308-b660-47a1-b884-27b37f91115a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:54.931065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104308-b660-47a1-b884-27b37f91115a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:54.931345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104308-b660-47a1-b884-27b37f91115a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:54.942291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104308-b660-47a1-b884-27b37f91115a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-78104308-b660-47a1-b884-27b37f91115a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:54.947433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:54.948525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:54.949657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 606/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:54.955542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:55.075814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f279bb8-d625-4961-a4ec-0e92050ab31d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:55.111936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:55.112550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:55.112833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:55.113092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:55.113828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:55.114128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:55.115519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:55.124063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 575/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:55.130482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104308-b660-47a1-b884-27b37f91115a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:55.163408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:55.237933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104308-b660-47a1-b884-27b37f91115a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:55.273491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:55.274128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:55.274410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:55.274701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] PATH_INFO: `/v3/users/e7bba0f885e84ea4869632467224ff80` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:55.287939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=e7bba0f885e84ea4869632467224ff80)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:55.289376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:55.291045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 607/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:55 2022] GET /identity/v3/users/e7bba0f885e84ea4869632467224ff80 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:55.297521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:55.359721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:55.409182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56272c71-b40f-4e8d-bbd7-c74ca0c16d86 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:55.445055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:55.445685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:55.445970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:55.446248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:55.519627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:55.521102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:55.585366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:55.633478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 576/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:01:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:01:55.639810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:55.736855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:55.758756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15374f4-212b-4f94-b25d-7354a2105ce1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:55.793259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd604e9-fd78-4cfa-bc94-75d454e348c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:55.793899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd604e9-fd78-4cfa-bc94-75d454e348c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:55.794181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd604e9-fd78-4cfa-bc94-75d454e348c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:55.794450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd604e9-fd78-4cfa-bc94-75d454e348c3 None admin] PATH_INFO: `/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:55.795667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd604e9-fd78-4cfa-bc94-75d454e348c3 None admin] RBAC: Authorizing `identity:get_project(project_id=f1ce2249202c4ca6ad4dde393ebf933c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:55.797086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd604e9-fd78-4cfa-bc94-75d454e348c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:55.798753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 608/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:55 2022] GET /identity/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:55.808819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:55.922010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6940b-2bd8-4cf1-9ab3-2ee9ea14d8df None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:55.959031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:55.959792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:55.960076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:55.960359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:55.961212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=f1ce2249202c4ca6ad4dde393ebf933c, user.id=e7bba0f885e84ea4869632467224ff80) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:55.961454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:55.962908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b0317-a89d-490b-a671-e92c925875b9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:55.968359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 577/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:01:55 2022] GET /identity/v3/role_assignments?user.id=e7bba0f885e84ea4869632467224ff80&scope.project.id=f1ce2249202c4ca6ad4dde393ebf933c&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:57.924246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161b237-5028-4b8e-8951-a1e574ba35e9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:57.924879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161b237-5028-4b8e-8951-a1e574ba35e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:57.924879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161b237-5028-4b8e-8951-a1e574ba35e9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:57.926638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 609/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:01:57.932593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:57.933081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:57.933207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:57.953021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:01:57.954280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:01:57.969113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:58.028951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 578/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:01:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:01:58.051049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161b237-5028-4b8e-8951-a1e574ba35e9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:58.162618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161b237-5028-4b8e-8951-a1e574ba35e9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:58.201830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:58.202410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:58.202705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:58.203015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:58.214493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:01:58.219468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:01:58.220498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:01:58.226399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 610/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:01:58.227316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:58.336534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d97bac-add9-4fbd-a23a-7e56c8cd9b8c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:58.385255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:58.385894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:58.386176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:58.386459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:58.387220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:01:58.387535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:58.388916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:58.397432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 579/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:58.405630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:58.424759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:58.516590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fb253-3b04-404f-8e04-cf34eb7a4955 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:58.553198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:58.553834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:58.554118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:58.554392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] PATH_INFO: `/v3/users/e7bba0f885e84ea4869632467224ff80` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:58.555654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] RBAC: Authorizing `identity:get_user(user_id=e7bba0f885e84ea4869632467224ff80)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:58.557096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:58.558688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 611/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:01:58 2022] GET /identity/v3/users/e7bba0f885e84ea4869632467224ff80 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:58.567921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:58.582044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:58.676667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514badc-df55-4053-a7bb-30b211d49ef1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:58.711804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3965d-4ef2-4102-b578-03a10ff636dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:58.712447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3965d-4ef2-4102-b578-03a10ff636dd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:58.712734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3965d-4ef2-4102-b578-03a10ff636dd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:58.713006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3965d-4ef2-4102-b578-03a10ff636dd None admin] PATH_INFO: `/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:58.714218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3965d-4ef2-4102-b578-03a10ff636dd None admin] RBAC: Authorizing `identity:get_project(project_id=f1ce2249202c4ca6ad4dde393ebf933c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:58.715698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3965d-4ef2-4102-b578-03a10ff636dd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:58.717383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 580/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:01:58 2022] GET /identity/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:01:58.726744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:01:58.812937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:01:58.832145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8404-6162-4267-9a41-cd0dba59a182 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:01:58.867025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30e289-bdd9-459a-a351-b0fabe78ffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:01:58.867750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30e289-bdd9-459a-a351-b0fabe78ffdc None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:01:58.868043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30e289-bdd9-459a-a351-b0fabe78ffdc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:01:58.868320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30e289-bdd9-459a-a351-b0fabe78ffdc None admin] PATH_INFO: `/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c/users/e7bba0f885e84ea4869632467224ff80/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:01:58.870397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30e289-bdd9-459a-a351-b0fabe78ffdc None admin] RBAC: Authorizing `identity:create_grant(project_id=f1ce2249202c4ca6ad4dde393ebf933c, user_id=e7bba0f885e84ea4869632467224ff80, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:01:58.871896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30e289-bdd9-459a-a351-b0fabe78ffdc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:01:58.886731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 612/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:01:58 2022] PUT /identity/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c/users/e7bba0f885e84ea4869632467224ff80/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:02:00.842116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7925716-a9f2-44f3-90d1-f62a32973f3f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:00.842660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7925716-a9f2-44f3-90d1-f62a32973f3f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:00.842660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7925716-a9f2-44f3-90d1-f62a32973f3f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:00.843929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 581/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:02:00.849429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:00.849840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:00.849980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:00.869458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:02:00.870668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:02:00.942554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 613/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:02:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:02:00.966030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7925716-a9f2-44f3-90d1-f62a32973f3f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:01.049798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:01.073775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7925716-a9f2-44f3-90d1-f62a32973f3f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:01.107219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:01.107781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:01.108060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:01.108333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:01.119406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:02:01.124316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:02:01.125365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:02:01.126211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 582/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:02:01.132255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:01.205570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:01.237890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352119fc-9306-4b2f-a9fc-f20f8289918e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:01.273730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:01.274361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:01.274648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:01.274957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:01.275688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:02:01.276010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:01.277397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:01.285842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 614/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:02:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:02:01.292053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:01.401254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c02c86-0140-4b47-a7ec-b1e1493be7a0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:01.436541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:01.437180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:01.437458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:01.437729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] PATH_INFO: `/v3/users/e7bba0f885e84ea4869632467224ff80` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:01.438952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] RBAC: Authorizing `identity:get_user(user_id=e7bba0f885e84ea4869632467224ff80)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:01.440388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:01.442000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 583/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jul 28 01:02:01 2022] GET /identity/v3/users/e7bba0f885e84ea4869632467224ff80 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:02:01.448227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:01.554516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b250145-5fa1-49fb-ac74-778f6770602f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:01.590869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:01.591542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:01.591821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:01.592099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:01.594881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:01.662964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:01.664339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:01.768490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 615/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:02:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:02:01.776185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:01.785922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:01.883768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef794-6889-4209-8d21-972bcadd4776 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:01.919385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db042882-aa5e-4ff5-ac4d-4a93c9498130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:01.919933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db042882-aa5e-4ff5-ac4d-4a93c9498130 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:01.920182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db042882-aa5e-4ff5-ac4d-4a93c9498130 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:01.920458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db042882-aa5e-4ff5-ac4d-4a93c9498130 None admin] PATH_INFO: `/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:01.921666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db042882-aa5e-4ff5-ac4d-4a93c9498130 None admin] RBAC: Authorizing `identity:get_project(project_id=f1ce2249202c4ca6ad4dde393ebf933c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:01.923230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db042882-aa5e-4ff5-ac4d-4a93c9498130 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:01.924911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 584/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jul 28 01:02:01 2022] GET /identity/v3/projects/f1ce2249202c4ca6ad4dde393ebf933c => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:02:01.931552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:01.967345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:02.041430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd81d2e-c625-4be2-8546-f27075ff2b10 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:02.077335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:02.077963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:02.078241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:02.078511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:02.079333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=f1ce2249202c4ca6ad4dde393ebf933c, user.id=e7bba0f885e84ea4869632467224ff80) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:02:02.079645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:02.081070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35981fe6-f3b5-4466-ac65-85438be8db47 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:02.087020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 616/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:02:01 2022] GET /identity/v3/role_assignments?user.id=e7bba0f885e84ea4869632467224ff80&scope.project.id=f1ce2249202c4ca6ad4dde393ebf933c&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:02:11.914439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=63 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:12.084211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=51 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:57.953612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22acc1-3103-4e61-9c58-62a0830b8e07 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:57.954243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22acc1-3103-4e61-9c58-62a0830b8e07 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:57.954243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22acc1-3103-4e61-9c58-62a0830b8e07 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:57.955589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 585/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:02:57.962883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:57.962883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:57.963898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:57.984420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:02:57.985736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:02:58.072471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 617/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:02:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:02:58.098034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22acc1-3103-4e61-9c58-62a0830b8e07 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:58.245137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22acc1-3103-4e61-9c58-62a0830b8e07 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:58.290504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:58.291208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:58.291483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:58.291876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:58.306224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:58.307751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:02:58.334955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:02:58.336056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:02:58.340082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:02:58.340082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:02:58.340082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:02:58.340082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:02:58.340082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 586/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:02:58.343536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:58.476538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790eaf4-f5cc-4eb5-9427-87e620078333 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:58.524360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:58.532050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:58.532802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:58.533088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:58.533482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:58.534440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:02:58.534955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:58.536508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcd127-efa3-4e18-9d61-41a00bbd0954 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:58.551430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 618/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:02:58.557733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:02:58.632422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:02:58.691746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3d8d5-5437-4a5f-b84e-6e964dd055f0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:02:58.743387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190b74a-d9c8-4070-9109-07ffeb1f8431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:02:58.744325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190b74a-d9c8-4070-9109-07ffeb1f8431 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:02:58.744774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190b74a-d9c8-4070-9109-07ffeb1f8431 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:02:58.745211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190b74a-d9c8-4070-9109-07ffeb1f8431 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:02:58.746086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190b74a-d9c8-4070-9109-07ffeb1f8431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:02:58.748082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190b74a-d9c8-4070-9109-07ffeb1f8431 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:02:58.773224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 587/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jul 28 01:02:58 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:03:01.037290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d809180-5762-4c33-8613-6e61b3dfcb68 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:01.037837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d809180-5762-4c33-8613-6e61b3dfcb68 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:01.038045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d809180-5762-4c33-8613-6e61b3dfcb68 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:01.039632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 619/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:01.045323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:01.045842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:01.045955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:01.065175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:01.066490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:01.141475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 588/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:01.165779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d809180-5762-4c33-8613-6e61b3dfcb68 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:01.289659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d809180-5762-4c33-8613-6e61b3dfcb68 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:01.341395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:01.342122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:01.342544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:01.342850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:01.362871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:03:01.363415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:01.364401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:03:01.365578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:01.368184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 620/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:03:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:03:01.374404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:01.421511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:01.496502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f364e-dc78-4c3b-84c0-fccc6bfdbe8b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:01.542481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:01.543066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:01.543483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:01.543831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:01.544967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:01.545357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:01.546943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:01.563732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 589/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:03:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:01.565091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:01.649201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:01.712429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bea93-63e4-42f6-86d5-3fdc28f55faf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:01.757201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:01.758165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:01.758570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:01.759179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:01.774613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:01.776264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:01.794559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:01.795749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:01.796849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:01.796849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:01.796849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:01.796849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:01.798048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 621/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:01.808555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:01.929226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:01.933353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ff1e5-312e-4ed2-b746-59091aad859b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:01.973147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:01.973810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:01.974099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:01.974415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:01.975373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:01.976592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:01.977463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:01.991653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 590/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:01.998148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:02.117073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e7a4a-bcb9-4fcf-917d-f95a1dab7b42 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:02.118917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:02.166048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:02.167713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:02.167713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:02.167713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:02.177310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:02.179016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:03:02.195193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:02.196506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:02.197687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:02.199224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 622/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:02.205712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:02.327880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacff2a-20fd-46dd-8641-6890e149752e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:02.370014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:02.370651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:02.371013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:02.371290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:02.372179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:02.372557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:02.374102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:02.397550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 591/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 01:03:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:02.403873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:02.541001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95256e8-a12a-4862-8384-075410e81959 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:02.582904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:02.583862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:02.584297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:02.584742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:02.624403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:02.670022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:02.672176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:02.797099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 623/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:03:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 01:03:02.803310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:02.825387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:02.934680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102205a-51db-4c1e-a1d5-7637e40a72b5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:02.974124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:02.974919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:02.975230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:02.975524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:02.988272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:02.990525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:03.008922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:03.010070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:03.011099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:03.011099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:03.011099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:03.011099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:03.011099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 592/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:03.015961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:03.088691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:03.134103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5af0-e944-49c0-9071-26b2ee65286d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:03.175413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:03.176364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:03.176862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:03.177318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:03.178248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:03.178702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:03.180780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:03.196561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 624/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:03:03.202137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:03.290191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:03.334245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acebc5-8091-4532-9e0c-ea16c1f33927 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:03.385902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:03.386997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:03.387485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:03.387949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:03.399506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:03.401959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:03:03.426726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:03.428591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:03.431159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 593/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:03.436501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:03.555411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713a012-2580-4bb3-99de-3874f9419cc6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:03.580122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:03.598865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:03.599583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:03.600020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:03.600344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:03.601149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:03.601459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:03.605392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24393f-b4c7-4618-b7cf-9aa9adb412e6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:03.627842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 625/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:03:03.629499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:03.757770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a473a8a-5ea5-4712-8c5e-778f97e39236 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:03.796435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:03.804345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:03.805131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:03.805425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:03.805704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:03.806520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=45f29692bb444492a58051889f4f44b4) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:03.806879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:03.808816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd4abf-650e-4948-b4bb-ba1a351f0f1e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:03.823066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 594/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:03:03 2022] GET /identity/v3/role_assignments?user.id=45f29692bb444492a58051889f4f44b4&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:03:06.099045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8529e50-6955-43f9-a2a8-9ecf7634aa95 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:06.100196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8529e50-6955-43f9-a2a8-9ecf7634aa95 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:06.100196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8529e50-6955-43f9-a2a8-9ecf7634aa95 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:06.101061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 626/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:06.107011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:06.107490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:06.107594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:06.127390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:06.128594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:06.205065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 595/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:06.231876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8529e50-6955-43f9-a2a8-9ecf7634aa95 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:06.375661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8529e50-6955-43f9-a2a8-9ecf7634aa95 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:06.414442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:06.415143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:06.415470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:06.415782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:06.428709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:03:06.434104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:06.435673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:06.436924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 627/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:03:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:06.442178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:06.580272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bffb40-159e-4315-abfd-95257a646805 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:06.606028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:06.618196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:06.618812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:06.619160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:06.619432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:06.620159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:06.620507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:06.621899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:06.634261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 596/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:03:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:06.640138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:06.690261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:06.752536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f17a6-5862-45ae-9732-5bc1a0baf784 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:06.790770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:06.791459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:06.791732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:06.792010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:06.802515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:06.804077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:06.819616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:06.820693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:06.822274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:06.822274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:06.822274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:06.822274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:06.822274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 628/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:06.826765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:06.942107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb0e1d-79f2-4f95-9579-1885a40ff48e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:06.982862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:06.982862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:06.982862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:06.982862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:06.983352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:06.983352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:06.983352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:06.996898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 597/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:07.003406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:07.103793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:07.111277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72fc46-1c88-4e7c-81e0-00f36ddf5d62 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:07.147308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:07.147974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:07.148264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:07.148536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:07.157610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:07.159471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:03:07.173778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:07.174994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:07.176338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 629/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:07.181978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:07.244375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:07.303712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459c67-9aae-4a73-8d77-4651697ba0c9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:07.342268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:07.342943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:07.343233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:07.343529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:07.344495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:07.344816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:07.346514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:07.371561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 598/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 01:03:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:07.372882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:07.489444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834cecc-63e7-4c7c-8e65-2ac62c73dbd8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:07.529428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:07.529983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:07.530272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:07.530532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:07.543628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:07.545313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:07.562060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:07.563240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:07.564351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:07.564351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:07.564351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:07.564351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:07.564351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 630/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:07.570303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:07.676092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a34000-472a-4b8a-baff-eb6d90459a28 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:07.678626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:07.713846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:07.714483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:07.714770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:07.715093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:07.715800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:07.716109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:07.717494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:07.730487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 599/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:07.736636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:07.746828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:07.848487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7ffd2-5bd5-45bb-938e-af26e8dc89f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:07.888175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:07.888840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:07.889126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:07.889405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:07.896522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:07.897986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:03:07.909742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:07.910922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:07.912121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 631/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:03:07.917048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:08.020574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351e1d2-225f-42d1-b3b6-5d57cd30b081 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:08.057474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34845813-c67e-40f5-9442-509768325db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:08.058483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34845813-c67e-40f5-9442-509768325db1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:08.058966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34845813-c67e-40f5-9442-509768325db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:08.059247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34845813-c67e-40f5-9442-509768325db1 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:08.060019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34845813-c67e-40f5-9442-509768325db1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:08.060328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34845813-c67e-40f5-9442-509768325db1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:08.061753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34845813-c67e-40f5-9442-509768325db1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:08.080092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 600/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:08.081769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:08.152170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:08.196316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd1ebe-bbe4-4e3a-bb80-cc644a1b6a75 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:08.239066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5730b-46cf-4213-bb86-833e882545ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:08.239794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5730b-46cf-4213-bb86-833e882545ec None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:08.240149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5730b-46cf-4213-bb86-833e882545ec None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:08.240483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5730b-46cf-4213-bb86-833e882545ec None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/45f29692bb444492a58051889f4f44b4/roles/12ea23231e5b4feba4d1270e89f737d9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:08.260244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5730b-46cf-4213-bb86-833e882545ec None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=45f29692bb444492a58051889f4f44b4, role_id=12ea23231e5b4feba4d1270e89f737d9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:08.261695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5730b-46cf-4213-bb86-833e882545ec None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:08.275060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 632/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:03:08 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/45f29692bb444492a58051889f4f44b4/roles/12ea23231e5b4feba4d1270e89f737d9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:03:10.300475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4892-aed0-4888-842d-cfdd4545c01d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:10.301020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4892-aed0-4888-842d-cfdd4545c01d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:10.301020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4892-aed0-4888-842d-cfdd4545c01d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:10.302463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 601/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:10.307923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:10.308337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:10.308506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:10.327621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:10.328907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:10.400498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 633/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:03:10.424702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4892-aed0-4888-842d-cfdd4545c01d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:10.455546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:10.538341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4892-aed0-4888-842d-cfdd4545c01d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:10.582190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:10.582749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:10.583087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:10.583365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:10.594991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:03:10.600109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:10.601209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 01:03:10.602527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:10.606769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 602/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jul 28 01:03:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:10.607962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:10.659682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:10.719208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b76c1-a9d4-4725-b9b4-de1419e5f156 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:10.756147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:10.756818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:10.757139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:10.757382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:10.758115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:10.758425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:10.759849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:10.768363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 634/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jul 28 01:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:10.774558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:10.870281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:10.891804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafcadd-1f9e-4340-8c78-cc99e102c716 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:10.929859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:10.932062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:10.932440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:10.932633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:10.945443 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:10.947100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:10.968222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:10.969256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:10.970128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:10.970128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:10.970128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:10.970128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:10.970128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 603/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:10.976170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:11.086336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857d23e-92d9-40a7-bfad-ace516367d45 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:11.095292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:11.121339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:11.121898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:11.122259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:11.122528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:11.123311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:11.123619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:11.125050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:11.138376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 635/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:11.144423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:11.255411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05788fcf-69d9-419d-9000-7951caf2256f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:11.291229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:11.291913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:11.292167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:11.292498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:11.295581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:11.301437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:11.302960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:03:11.316337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:11.317644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:11.318810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 604/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:11.323828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:11.429086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf3e0d-5bc7-47f4-8cd4-e4a910b5cc55 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:11.467706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:11.468324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:11.468605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:11.468879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:11.469684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:11.469994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:11.471460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:11.496048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 636/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 01:03:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:11.498280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:11.613349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9d5c3-2f6a-4102-90f3-a878ab78edb4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:11.658154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:11.659002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:11.659317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:11.659624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:11.725943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:11.736060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:11.737651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:11.855449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 605/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:03:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:03:11.861992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:11.894364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:11.975932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec041d-db2d-4338-bfc6-3e2c623be014 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:12.014031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:12.014596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:12.014908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:12.015184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:12.025045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:12.026486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:12.041964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:12.043206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:12.044371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:12.044371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:12.044371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:12.044371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:12.044371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 637/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:12.049686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:12.153506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:12.160033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39b3d5-4f7f-40e0-9d9f-f12dd23e07c3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:12.203764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:12.204469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:12.204893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:12.205235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:12.206091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:12.206433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:12.208150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:12.223933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 606/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:03:12.230129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:12.325414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:12.351831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ed21a-d680-4dcf-8345-d442b82303fe None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:12.391683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:12.392543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:12.392885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:12.393227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:12.401229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:12.402778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:03:12.418527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:12.419697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:12.420935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 638/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:12.426477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:12.537020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293bece-455c-49f1-90ca-ef7fc4fd68b5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:12.573438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:12.574464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:12.574948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:12.575238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:12.576240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:12.576692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:12.578554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178b49e-8c9f-4183-96ea-117698d32ee6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:12.591820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 607/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:12.598243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:12.731749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebc73b-83f1-4eaa-9fe2-0edbd406dc8a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:12.768061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:12.768691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:12.768976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:12.769252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:12.770037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] RBAC: Adding query filter params (role.id=12ea23231e5b4feba4d1270e89f737d9, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=45f29692bb444492a58051889f4f44b4) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:12.770367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:12.771844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece683-21e6-477d-90d8-e7bd67ccc7f1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:12.778644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 639/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:03:12 2022] GET /identity/v3/role_assignments?user.id=45f29692bb444492a58051889f4f44b4&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=12ea23231e5b4feba4d1270e89f737d9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:15.089221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ac597-b9a8-42e0-9602-e0c3efb1fe11 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:15.090037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ac597-b9a8-42e0-9602-e0c3efb1fe11 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:15.090037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ac597-b9a8-42e0-9602-e0c3efb1fe11 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:15.091186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 608/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:15.097155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:15.097625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:15.097772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:15.104653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:15.118474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:15.119681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:15.204854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 640/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:15.232200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ac597-b9a8-42e0-9602-e0c3efb1fe11 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:15.255315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:15.352815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ac597-b9a8-42e0-9602-e0c3efb1fe11 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:15.397373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:15.398230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:15.398577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:15.398988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:15.411761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:03:15.416784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:15.418060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:15.419075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 609/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:03:15.424372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:15.541425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6d85-0334-49d9-85a6-9be8e49fc024 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:15.579073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:15.585822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:15.586861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:15.587161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:15.587561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:15.588288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:15.588595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:15.589968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:15.600586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 641/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:15.606716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:15.699360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:15.725356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3965352-c1df-446c-a1f0-e91ba0c9a797 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:15.770661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:15.771560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:15.771937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:15.772326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:15.784783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:15.786221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:15.803068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:15.805169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:15.806289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:15.806289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:15.806289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:15.806289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:15.806289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 610/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:15.810265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:15.924717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d627e-d708-4e22-9721-0840d045ffa9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:15.963345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb051abd-b116-493f-859a-956256946f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:15.964128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb051abd-b116-493f-859a-956256946f43 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:15.964416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb051abd-b116-493f-859a-956256946f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:15.964726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb051abd-b116-493f-859a-956256946f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:15.965465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb051abd-b116-493f-859a-956256946f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:15.965759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb051abd-b116-493f-859a-956256946f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:15.967371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb051abd-b116-493f-859a-956256946f43 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:15.986274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 642/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:15.987569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:16.102545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3909-ec94-45e2-b287-d440a2564610 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:16.132447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:16.141404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:16.142074 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:16.142385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:16.142651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:16.151312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:16.152888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:03:16.165022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:16.166145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:16.167145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 611/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:16.174417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb051abd-b116-493f-859a-956256946f43 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:16.247269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:16.298345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb051abd-b116-493f-859a-956256946f43 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:16.336532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:16.337246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:16.337533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:16.337880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:16.338689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:16.339022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:16.340454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:16.359272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 643/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 01:03:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:03:16.365391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:16.480645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b78a5b-a8a1-4bfc-8028-c0db4da0d7f6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:16.515329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:16.515933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:16.516212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:16.516483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:16.579234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:16.591944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:16.593569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:16.701649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 612/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:03:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:03:16.707913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:16.815951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdca1a-ea30-4d06-a3c8-b19314712772 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:16.829560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:16.857603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:16.858668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:16.859177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:16.859642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:16.871826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:16.873354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:16.889427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:16.890601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:16.891739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:16.891739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:16.891739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:16.891739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:16.891739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 644/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:16.896911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:16.993380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:17.009120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59c515-84af-439f-927d-04f601b857a4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:17.047831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:17.048522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:17.048838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:17.049137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:17.049882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:17.050199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:17.051958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:17.073115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 613/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:17.075031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:17.211828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f1d1-5b57-4821-8096-741d09333d6e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:17.252981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:17.253640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:17.253930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:17.254210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:17.261343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:17.262793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:03:17.275108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:17.276317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:17.277317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 645/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:17.282911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:17.418055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098073f-990b-4f43-9465-b82d82969fef None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:17.474353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:17.474985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:17.475273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:17.475706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:17.476607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:17.476607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:17.478160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14414849-cfbc-4921-bc3c-e0db4e3be3ae None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:17.495242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 614/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:17.503223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:17.532896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:17.620970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4f0d6-d7d8-44ad-8b85-1de50b616e43 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:17.658611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:17.659313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:17.659560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:17.659844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:17.660687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=45f29692bb444492a58051889f4f44b4) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:17.661018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:17.662449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa72a6-2679-459e-913d-db2e58475edc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:17.668345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 646/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:03:17 2022] GET /identity/v3/role_assignments?user.id=45f29692bb444492a58051889f4f44b4&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=132431a386d641a8aaeba0496b601812 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:19.691146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ed398-eac6-49e7-9189-f326803be7ed None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:19.692035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ed398-eac6-49e7-9189-f326803be7ed None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:19.692035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ed398-eac6-49e7-9189-f326803be7ed None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:19.694231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 615/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:19.702739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:19.703501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:19.703594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:19.730475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:19.731623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:19.808163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 647/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:19.834273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ed398-eac6-49e7-9189-f326803be7ed None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:19.846404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:19.952697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ed398-eac6-49e7-9189-f326803be7ed None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:19.994782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:19.995380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:19.995672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:19.995945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:20.007765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:03:20.012665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:20.013734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:20.014634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 616/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:03:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:20.020793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:20.040697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:20.132990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0224-8939-4a4e-8d8b-ec009ac233fe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:20.176086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:20.177048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:20.177459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:20.177798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:20.178790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:20.179220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:20.180989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:20.190808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 648/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:20.198702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:20.282676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:20.319869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f431769-fdd8-43d1-8a40-37e16409eee2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:20.358194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:20.358867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:20.359154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:20.359428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:20.370147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:20.371640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:20.387057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:20.388054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:20.388988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:20.388988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:20.388988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:20.388988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:20.388988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 617/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 01:03:20.394396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:20.484320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:20.505002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cc953-fb53-4e04-89eb-ff7581bc593e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:20.539738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:20.540306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:20.540595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:20.540873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:20.541577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:20.541892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:20.543291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:20.555236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 649/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:20.561859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:20.667685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d36a8c-ca39-4f2e-bd38-aeabdabb732b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:20.688181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:20.703032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:20.703679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:20.703940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:20.704199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:20.711840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:20.713272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:03:20.724450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:20.725759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:20.726708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 618/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:20.734158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:20.842943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269edcb5-8b69-48d3-83b9-b5f7182af404 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:20.878682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:20.879310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:20.879968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:20.880277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:20.881091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:20.881402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:20.882859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:20.891665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:20.902020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 650/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 01:03:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:20.908041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:21.031533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33983a-7477-4453-a178-19263d1a4439 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:21.074614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:21.075300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:21.075589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:21.075869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:21.087921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:21.089373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:21.104946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:21.105945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:21.106979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:21.106979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:21.106979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:21.106979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:21.106979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 619/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:21.112794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:21.236140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931eb4c-32f0-4ca6-ab12-811389f31512 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:21.272634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:21.273260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:21.273548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:21.273822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:21.274534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:21.274864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:21.276240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:21.287737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 651/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:21.296663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:21.338291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:21.409797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc01fa-4a05-4a46-b39b-ddaada9ff449 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:21.445468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:21.446115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:21.446403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:21.446682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:21.454517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:21.456049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:03:21.468657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:21.469672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:21.470713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 620/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:21.476406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:21.549707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:21.596418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d1d43-53d2-4f7c-8f58-8732837d2549 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:21.633332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:21.633968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:21.634253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:21.634537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:21.635332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:21.635653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:21.637073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88825b-9a10-4f33-b46e-91115e7522a5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:21.649475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 652/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:21.656372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:21.769578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:21.779851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade62b-cb34-44b1-9301-0fa34ac8381e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:21.815370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89bf36-6dfd-4e2b-9193-56b484c2258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:21.816038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89bf36-6dfd-4e2b-9193-56b484c2258e None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:21.816309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89bf36-6dfd-4e2b-9193-56b484c2258e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:21.816582 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89bf36-6dfd-4e2b-9193-56b484c2258e None admin] PATH_INFO: `/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/45f29692bb444492a58051889f4f44b4/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:21.818619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89bf36-6dfd-4e2b-9193-56b484c2258e None admin] RBAC: Authorizing `identity:create_grant(project_id=ae4e155b9bf04e6ba575e5d36a86fd2b, user_id=45f29692bb444492a58051889f4f44b4, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:21.820117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89bf36-6dfd-4e2b-9193-56b484c2258e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:21.838907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 621/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jul 28 01:03:21 2022] PUT /identity/v3/projects/ae4e155b9bf04e6ba575e5d36a86fd2b/users/45f29692bb444492a58051889f4f44b4/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:03:23.835449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac723a-8f6b-4962-a3a2-bab0092e3f7d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:23.835994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac723a-8f6b-4962-a3a2-bab0092e3f7d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:23.835994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac723a-8f6b-4962-a3a2-bab0092e3f7d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:23.837291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 653/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:23.842873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:23.843366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:23.843456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:23.863508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:23.864762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:23.937153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 622/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:23.961684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac723a-8f6b-4962-a3a2-bab0092e3f7d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:24.078088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac723a-8f6b-4962-a3a2-bab0092e3f7d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:24.097198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:24.115909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:24.116554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:24.116839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:24.117116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:24.127395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 01:03:24.132825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:24.133886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:24.135168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 654/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jul 28 01:03:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:24.140306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:24.234653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:24.256873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d539fd-cd7d-4d4b-96d0-ba90690a0baa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:24.291762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:24.292383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:24.292662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:24.292939 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:24.293664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:24.293964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:24.295362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:24.303475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 623/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jul 28 01:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:24.312778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:24.430342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c96125-9ea2-461a-b4bf-8f80e6f3fa05 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:24.472028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:24.472682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:24.472968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:24.473246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:24.484724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:24.486138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:24.501224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:24.502247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:24.503250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:24.503250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:24.503250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:24.503250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:24.503250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 655/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:24.508855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:24.612715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6e31-a395-444c-ba7d-e653ad2b9eb2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:24.652541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:24.653173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:24.653460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:24.653732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:24.654435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:24.654748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:24.656160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:24.670547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 624/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:24.679026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:24.712925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:24.787700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc1cf6-c08e-449d-9c88-02c442941af6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:24.828203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:24.828846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:24.829137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:24.829422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:24.837417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:24.838877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 01:03:24.850565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:24.851681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:24.853595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 656/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:24.861021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:24.868825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:24.977780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57139b1a-7d39-43b6-8aea-4fa573ab9677 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:25.012732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:25.013344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:25.013622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:25.013893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:25.014688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:25.015033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:25.016455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:25.036240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 625/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jul 28 01:03:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:25.045920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:25.150185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:25.162091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e534d-602d-41ff-9f3b-1064917d8e94 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:25.206445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:25.207106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:25.207400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:25.207761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:25.280289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:25.281665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:25.383695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:25.394828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 657/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jul 28 01:03:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 01:03:25.403515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:25.472527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:25.512618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0fb7a-a4c9-40c7-a586-3480ddffd489 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:25.547906 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:25.548477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:25.548761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:25.549038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:25.560219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:25.561676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 01:03:25.577355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:25.578422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:25.579710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 01:03:25.579710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 01:03:25.579710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 01:03:25.579710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:25.579710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 626/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:25.584538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:25.690631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a08be-b3f5-472c-872d-cbba2c3eed70 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:25.726660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:25.727332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:25.727617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:25.727905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:25.728597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:25.728904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:25.730270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:25.742582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 658/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jul 28 01:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:25.751914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:25.859223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965211a-e839-46c6-8660-e687a2f7d875 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:25.873782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:25.902295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:25.902984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:25.903272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:25.903552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:25.910323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:25.911805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 01:03:25.923199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:25.924268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Jul 28 01:03:25.925200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 627/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jul 28 01:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:25.933004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:25.991065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:26.055850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4c169-4f9a-40ca-9d42-ad8265ce7c81 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:26.093182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:26.093815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:26.094100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:26.094382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:26.095179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:26.095490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:26.096942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533a350-98c3-41c4-80b3-69732fb3d917 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:26.109669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 659/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jul 28 01:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:26.119892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:26.249598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1a3a-2d01-4f60-be40-db4138b1e174 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:26.288695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:26.289322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:26.289603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:26.289881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:26.290673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] RBAC: Adding query filter params (role.id=132431a386d641a8aaeba0496b601812, scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b, user.id=45f29692bb444492a58051889f4f44b4) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:26.291022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:26.292476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f9825-0a1d-44c7-a0b8-512fe66b17b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:26.299007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 628/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jul 28 01:03:26 2022] GET /identity/v3/role_assignments?user.id=45f29692bb444492a58051889f4f44b4&scope.project.id=ae4e155b9bf04e6ba575e5d36a86fd2b&role.id=132431a386d641a8aaeba0496b601812 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:28.300136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e26d47-93bd-4b02-857d-09b4b2406023 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:28.301034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e26d47-93bd-4b02-857d-09b4b2406023 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:28.301034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e26d47-93bd-4b02-857d-09b4b2406023 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:28.303410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 660/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:28.311331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:28.311803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:28.311909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:28.331719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:28.332980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:28.369574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:28.413712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 629/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 01:03:28.436275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e26d47-93bd-4b02-857d-09b4b2406023 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:28.545324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e26d47-93bd-4b02-857d-09b4b2406023 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:28.556069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:28.582513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:28.583204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:28.583491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:28.583766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:28.584407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:28.585758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:28.594627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:28.595913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:28.596802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 661/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:03:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:28.602854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:28.721386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dc141-4c88-4dc4-b2fd-8d4eb74a8658 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:28.761525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:28.762147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:28.762429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:28.762707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:28.763414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:28.763721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:28.765087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f06d-c2d6-4984-aa04-9efcc2622d9d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:28.770672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 630/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 01:03:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:28.780282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:28.892730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec64a7-4f40-42f6-81f9-6f8fabe74322 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:28.927809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:28.928496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:28.928785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:28.929062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:28.929749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:28.930046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:28.931477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f107f1-7588-4f2c-bfd9-e7846930fed9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:28.936030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 662/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 01:03:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:31.050130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33405c6-53ff-4ce2-8905-6bba9d844bdd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:31.050731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33405c6-53ff-4ce2-8905-6bba9d844bdd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:31.050731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33405c6-53ff-4ce2-8905-6bba9d844bdd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:31.052183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 631/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:31.058794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0224-ef3d-45db-b8c2-b5a04a27cc0f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:31.059649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0224-ef3d-45db-b8c2-b5a04a27cc0f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:31.059819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0224-ef3d-45db-b8c2-b5a04a27cc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:31.087025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-aa1d0224-ef3d-45db-b8c2-b5a04a27cc0f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:31.088239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aa1d0224-ef3d-45db-b8c2-b5a04a27cc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:31.122649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:31.166905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 663/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:31.198801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33405c6-53ff-4ce2-8905-6bba9d844bdd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:31.239237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:31.316010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33405c6-53ff-4ce2-8905-6bba9d844bdd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:31.363145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9d838-89ab-4c88-ae33-6b0782067e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:31.363775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9d838-89ab-4c88-ae33-6b0782067e68 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:31.364057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9d838-89ab-4c88-ae33-6b0782067e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:31.364334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9d838-89ab-4c88-ae33-6b0782067e68 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:31.365069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9d838-89ab-4c88-ae33-6b0782067e68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:31.366543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9d838-89ab-4c88-ae33-6b0782067e68 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:31.383831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 632/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jul 28 01:03:31 2022] POST /identity/v3/services => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:03:33.432408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7197c-5f6d-4f3f-82d8-639720cf15ae None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:33.432976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7197c-5f6d-4f3f-82d8-639720cf15ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:33.432976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7197c-5f6d-4f3f-82d8-639720cf15ae None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:33.434428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 664/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:33.440934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:33.441594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:33.441687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:33.467956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:33.469585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:33.570899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 633/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:33.595819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7197c-5f6d-4f3f-82d8-639720cf15ae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:33.728597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7197c-5f6d-4f3f-82d8-639720cf15ae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:33.766671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:33.767365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:33.767648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:33.767924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:33.768541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:33.770357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:33.779993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:33.781233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:33.782080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 665/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:03:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:33.787908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:33.808519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:33.910814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2daf98-4d9d-4b50-b994-f724fda526c9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:33.955147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:33.955808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:33.956094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:33.956375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:33.957065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:33.957361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:33.958766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:33.964991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 634/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 01:03:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:33.971942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:34.010424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:34.100480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c41dd8-cd7a-4259-bfdc-ed12662b801d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:34.147371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:34.148020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:34.148303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:34.148588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:34.149350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] RBAC: Adding query filter params (interface=public, service_id=2fe8a3ec5e6b48f69290c87a94f6f0d3, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:34.149668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:34.151073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae769b-878a-49a3-be71-ae0e13eb4582 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:34.156895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 666/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jul 28 01:03:33 2022] GET /identity/v3/endpoints?service_id=2fe8a3ec5e6b48f69290c87a94f6f0d3&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:34.164181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:34.274158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:34.294064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ca66-482f-40e1-ba9e-a0bb1356b91e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:34.331679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af2f01-0ad7-4191-87df-24344829eb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:34.332855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af2f01-0ad7-4191-87df-24344829eb3f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:34.332855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af2f01-0ad7-4191-87df-24344829eb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:34.332855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af2f01-0ad7-4191-87df-24344829eb3f None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:34.333473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af2f01-0ad7-4191-87df-24344829eb3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:34.335008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af2f01-0ad7-4191-87df-24344829eb3f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:34.341893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 635/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jul 28 01:03:34 2022] GET /identity/v3/services => generated 2191 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:03:36.408801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd667a-5cc3-4bed-87a3-6ddfe199592b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:36.409676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd667a-5cc3-4bed-87a3-6ddfe199592b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:36.409676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd667a-5cc3-4bed-87a3-6ddfe199592b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:36.411942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 667/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:03:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:03:36.422478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:36.423320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:36.423533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:36.445992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:03:36.447241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:03:36.516474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 636/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:03:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 01:03:36.539049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd667a-5cc3-4bed-87a3-6ddfe199592b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:36.645879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:36.654599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd667a-5cc3-4bed-87a3-6ddfe199592b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:36.690656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:36.691261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:36.691544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:36.691824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:36.692450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:36.693876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 01:03:36.702824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 01:03:36.703978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Jul 28 01:03:36.704979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 668/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jul 28 01:03:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 01:03:36.710294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:36.792581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:36.819375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd946ffa-ec12-4b59-b9eb-ee5424e144b1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:36.861863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:36.862495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:36.862781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:36.863085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:36.863777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:03:36.864084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:36.865455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902759c4-0f2d-47a5-878d-aa7205632e23 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:36.871323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 637/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jul 28 01:03:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:03:36.878172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:03:36.987877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357039fe-bf3a-4814-8981-21c8f593fe7f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:03:37.024149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfd81-1bf3-41f5-9671-5f708e8b9bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:03:37.024776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfd81-1bf3-41f5-9671-5f708e8b9bcc None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:03:37.025071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfd81-1bf3-41f5-9671-5f708e8b9bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:03:37.025343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfd81-1bf3-41f5-9671-5f708e8b9bcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:03:37.026056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bfd81-1bf3-41f5-9671-5f708e8b9bcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:03:37.027468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bfd81-1bf3-41f5-9671-5f708e8b9bcc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:03:37.051297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 669/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:03:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:03:46.868342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=30 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:03:47.046774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:40.905133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863276b-8d78-4efe-ba79-a8cfc37d1bca None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:40.906081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863276b-8d78-4efe-ba79-a8cfc37d1bca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:40.906525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863276b-8d78-4efe-ba79-a8cfc37d1bca None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:40.908145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 638/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:40.913987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883fe5-4d80-4354-9ef7-d85b6b123569 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:40.914868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883fe5-4d80-4354-9ef7-d85b6b123569 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:40.915303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883fe5-4d80-4354-9ef7-d85b6b123569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:41.040437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bc883fe5-4d80-4354-9ef7-d85b6b123569 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:41.042745 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bc883fe5-4d80-4354-9ef7-d85b6b123569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:41.262859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 670/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:41.304494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25675f36-d783-4e39-9b9d-100e6412d108 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:41.305106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25675f36-d783-4e39-9b9d-100e6412d108 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:41.305661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25675f36-d783-4e39-9b9d-100e6412d108 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:41.307423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 639/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 01:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:41.314227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df74d6e-5b70-4c89-a05c-ccc9b6cd8bb9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:41.314685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df74d6e-5b70-4c89-a05c-ccc9b6cd8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:41.315220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df74d6e-5b70-4c89-a05c-ccc9b6cd8bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:41.399404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5df74d6e-5b70-4c89-a05c-ccc9b6cd8bb9 None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:41.400647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5df74d6e-5b70-4c89-a05c-ccc9b6cd8bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:41.422599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:41.483369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 671/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 01:04:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:41.492544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25675f36-d783-4e39-9b9d-100e6412d108 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:41.494319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-25675f36-d783-4e39-9b9d-100e6412d108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:41.616026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25675f36-d783-4e39-9b9d-100e6412d108 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:41.698473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:41.699063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:41.699352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:41.699631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:41.701387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:41.759088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:41.820913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:04:41.822444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491337e-a6c8-4a7b-b5dd-c14271a22803 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:04:41.843967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 640/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:04:44.145945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9cfa7-7a17-4650-a713-99b1df616061 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:44.146512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9cfa7-7a17-4650-a713-99b1df616061 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:44.146512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9cfa7-7a17-4650-a713-99b1df616061 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:44.147931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 672/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:44.153527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0dde5-1e99-49b5-afe9-a734db1de10f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:44.154030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0dde5-1e99-49b5-afe9-a734db1de10f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:44.154127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0dde5-1e99-49b5-afe9-a734db1de10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:44.229742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0dc0dde5-1e99-49b5-afe9-a734db1de10f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:44.231005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0dc0dde5-1e99-49b5-afe9-a734db1de10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:44.349308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 641/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:44.388264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9cfa7-7a17-4650-a713-99b1df616061 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:44.390026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3de9cfa7-7a17-4650-a713-99b1df616061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:44.518515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9cfa7-7a17-4650-a713-99b1df616061 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:44.600374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:44.603173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:44.603912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:44.604267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:44.604732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:44.607207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:44.729882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:04:44.731554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609467b-3cb7-4357-a629-1272994aa1f2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:04:44.753236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 673/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:04:47.086432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db233178-a074-41f8-93a0-e481bd2dffb2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:47.087509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db233178-a074-41f8-93a0-e481bd2dffb2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:47.087509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db233178-a074-41f8-93a0-e481bd2dffb2 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:47.089566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 642/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:47.098345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07bff-d994-4ebf-b0c2-dd976ce6e9dc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:47.098818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07bff-d994-4ebf-b0c2-dd976ce6e9dc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:47.098956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07bff-d994-4ebf-b0c2-dd976ce6e9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:47.173636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1ec07bff-d994-4ebf-b0c2-dd976ce6e9dc None None] MFA Rules not processed for user `e7daa2b0f34a4966a673e1795e3cba54`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:47.174878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1ec07bff-d994-4ebf-b0c2-dd976ce6e9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:47.231453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:47.278646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 674/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:47.340654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040189a6-f9e9-47f6-9112-8471aa7bcbad None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:47.340946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040189a6-f9e9-47f6-9112-8471aa7bcbad None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:47.341295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040189a6-f9e9-47f6-9112-8471aa7bcbad None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:47.343462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 643/1317] 10.222.0.17 () {54 vars in 965 bytes} [Thu Jul 28 01:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:47.350453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35318996-5249-48bc-b29e-fb148408ce9f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:47.350826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35318996-5249-48bc-b29e-fb148408ce9f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:47.351258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35318996-5249-48bc-b29e-fb148408ce9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:47.431991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-35318996-5249-48bc-b29e-fb148408ce9f None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:47.433273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-35318996-5249-48bc-b29e-fb148408ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:47.519296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 675/1318] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Jul 28 01:04:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:47.528569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040189a6-f9e9-47f6-9112-8471aa7bcbad None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:47.530145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-040189a6-f9e9-47f6-9112-8471aa7bcbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:47.541239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:47.660526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040189a6-f9e9-47f6-9112-8471aa7bcbad None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:47.744338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:47.745017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:47.745303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:47.745581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:47.747787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:47.821005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:47.874857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:04:47.877139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d497d6-8f51-47a1-bb37-a3904684c198 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:04:47.899269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 644/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:04:50.899847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdd4cf-c269-4368-ad44-18763283503e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:50.900768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdd4cf-c269-4368-ad44-18763283503e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:50.900768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdd4cf-c269-4368-ad44-18763283503e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:50.902998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 676/1320] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:50.910882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756747f-3eac-43d6-926b-6f09b4b38a0b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:50.911354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756747f-3eac-43d6-926b-6f09b4b38a0b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:50.911462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756747f-3eac-43d6-926b-6f09b4b38a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:50.979962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6756747f-3eac-43d6-926b-6f09b4b38a0b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:50.981202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6756747f-3eac-43d6-926b-6f09b4b38a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:51.081486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 645/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:51.128734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdd4cf-c269-4368-ad44-18763283503e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:51.130669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9fbdd4cf-c269-4368-ad44-18763283503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:51.187091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:51.251811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdd4cf-c269-4368-ad44-18763283503e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:51.329985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:51.330617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:51.330926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:51.331209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:51.332870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:51.437657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:51.451354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:04:51.452800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf0f33-f9c5-4d20-bc5d-d342cb3e1ce5 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:04:51.471612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 677/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:04:54.224192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b12831-06f7-4420-8f49-2731dba4de73 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:54.224796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b12831-06f7-4420-8f49-2731dba4de73 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:54.224796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b12831-06f7-4420-8f49-2731dba4de73 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:54.226118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 646/1323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:54.231574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fcc32-e313-4c46-958f-b62c028bc941 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:54.232010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fcc32-e313-4c46-958f-b62c028bc941 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:54.232154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fcc32-e313-4c46-958f-b62c028bc941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:54.300726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4a9fcc32-e313-4c46-958f-b62c028bc941 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:54.301948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a9fcc32-e313-4c46-958f-b62c028bc941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:54.407032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 678/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:54.440727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b12831-06f7-4420-8f49-2731dba4de73 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:54.442465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1b12831-06f7-4420-8f49-2731dba4de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:54.466969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:54.566269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b12831-06f7-4420-8f49-2731dba4de73 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:54.647882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:54.648521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:54.648801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:54.649070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:54.650706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:54.724403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:54.766767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:04:54.768592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e52aa4-6975-446c-a833-a9d5f8e51195 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:04:54.789170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 647/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:04:56.859701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ce74-4c51-4fca-9551-cbbcbae69375 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:56.860247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ce74-4c51-4fca-9551-cbbcbae69375 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:56.860247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ce74-4c51-4fca-9551-cbbcbae69375 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:56.861547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 679/1326] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:56.867135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7ac9b-e2fd-43c4-b4b1-cbf6d682ed7b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:56.867592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7ac9b-e2fd-43c4-b4b1-cbf6d682ed7b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:56.867592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7ac9b-e2fd-43c4-b4b1-cbf6d682ed7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:56.937357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6de7ac9b-e2fd-43c4-b4b1-cbf6d682ed7b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:56.938577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6de7ac9b-e2fd-43c4-b4b1-cbf6d682ed7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:57.038423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 648/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:57.074562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ce74-4c51-4fca-9551-cbbcbae69375 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:57.076238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9da3ce74-4c51-4fca-9551-cbbcbae69375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:57.142095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:57.189496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ce74-4c51-4fca-9551-cbbcbae69375 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:57.266967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:57.267615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:57.267897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:57.268168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:57.269960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:57.385735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:57.388271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:04:57.389772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989a3fc-2885-4c6b-9694-f45283c5b6e3 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:04:57.410056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 680/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:04:59.483829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036903be-33db-4a97-b15b-aad93f20640f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:59.484350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036903be-33db-4a97-b15b-aad93f20640f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:59.484472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036903be-33db-4a97-b15b-aad93f20640f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:59.485804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 649/1329] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:04:59.491884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba167290-2954-4ecf-bebc-559e04fbe228 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:59.492397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba167290-2954-4ecf-bebc-559e04fbe228 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:59.492492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba167290-2954-4ecf-bebc-559e04fbe228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:59.569606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ba167290-2954-4ecf-bebc-559e04fbe228 None None] MFA Rules not processed for user `e7daa2b0f34a4966a673e1795e3cba54`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:04:59.570821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ba167290-2954-4ecf-bebc-559e04fbe228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:59.687311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 681/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:04:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:04:59.725245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036903be-33db-4a97-b15b-aad93f20640f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:04:59.726796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-036903be-33db-4a97-b15b-aad93f20640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:04:59.759966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:04:59.839696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036903be-33db-4a97-b15b-aad93f20640f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:04:59.922383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:04:59.923071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:04:59.923355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:04:59.923627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:04:59.925291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:00.006111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:00.039234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:00.040700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf257e-f5a3-4edd-a004-8203e73be782 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:00.075269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 650/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:04:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:02.324914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d212-19c3-44ad-8ed9-c4921a7fa538 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:02.325519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d212-19c3-44ad-8ed9-c4921a7fa538 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:02.325519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d212-19c3-44ad-8ed9-c4921a7fa538 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:02.326871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 682/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:02.333724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6542d03-e32e-4920-9e7e-1db758b22bc7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:02.334380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6542d03-e32e-4920-9e7e-1db758b22bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:02.334476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6542d03-e32e-4920-9e7e-1db758b22bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:02.416224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c6542d03-e32e-4920-9e7e-1db758b22bc7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:02.417341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c6542d03-e32e-4920-9e7e-1db758b22bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:02.535054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 651/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:02.574837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30d212-19c3-44ad-8ed9-c4921a7fa538 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:02.577128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5e30d212-19c3-44ad-8ed9-c4921a7fa538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:02.658740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:02.696398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30d212-19c3-44ad-8ed9-c4921a7fa538 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:02.901735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:02.902367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:02.902651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:02.902955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:02.904610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:03.020932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:03.022271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3bf8c-e4fa-449b-a0d0-7586594ae36b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:03.032578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:03.043067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 683/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:05.081977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523485f-bf74-4372-be03-9b122ea3d835 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:05.082905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523485f-bf74-4372-be03-9b122ea3d835 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:05.082905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523485f-bf74-4372-be03-9b122ea3d835 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:05.085184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 652/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:05.090669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e32b7-857f-42c3-84ce-38baadf52328 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:05.091130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e32b7-857f-42c3-84ce-38baadf52328 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:05.091241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e32b7-857f-42c3-84ce-38baadf52328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:05.160868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-249e32b7-857f-42c3-84ce-38baadf52328 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:05.162093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-249e32b7-857f-42c3-84ce-38baadf52328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:05.266717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 684/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:05.310125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523485f-bf74-4372-be03-9b122ea3d835 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:05.312715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1523485f-bf74-4372-be03-9b122ea3d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:05.365277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:05.439288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523485f-bf74-4372-be03-9b122ea3d835 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:05.521959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:05.522615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:05.522918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:05.523195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:05.524841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:05.620282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:05.648449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:05.649925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1c6ae-7e67-44f7-a00b-8e8878f36924 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:05.668357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 653/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:08.131366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dc33-44e4-40b6-8528-bd71cd41f035 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:08.132226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dc33-44e4-40b6-8528-bd71cd41f035 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:08.132226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480dc33-44e4-40b6-8528-bd71cd41f035 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:08.134450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 685/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:08.140048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba2d1-0ac7-45ae-9728-fdd7342b767f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:08.140574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba2d1-0ac7-45ae-9728-fdd7342b767f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:08.140574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba2d1-0ac7-45ae-9728-fdd7342b767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:08.211967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3eeba2d1-0ac7-45ae-9728-fdd7342b767f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:08.213224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3eeba2d1-0ac7-45ae-9728-fdd7342b767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:08.318379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 654/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:08.356530 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480dc33-44e4-40b6-8528-bd71cd41f035 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:08.358934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d480dc33-44e4-40b6-8528-bd71cd41f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:08.455471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:08.485050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480dc33-44e4-40b6-8528-bd71cd41f035 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:08.561995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:08.562622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:08.562933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:08.563203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:08.564819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:08.679101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:08.680528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d952c-b585-4f90-962b-9bbb8244a254 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:08.699182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 686/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:12.844511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb176468-b56a-44f9-8230-41a09b5f9fe6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:12.845048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb176468-b56a-44f9-8230-41a09b5f9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:12.845048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb176468-b56a-44f9-8230-41a09b5f9fe6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:12.846502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 655/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:12.851874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bb22-efc5-4483-8af8-74444d89d792 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:12.852352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bb22-efc5-4483-8af8-74444d89d792 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:12.852352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bb22-efc5-4483-8af8-74444d89d792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:12.858100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:12.920364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b9a9bb22-efc5-4483-8af8-74444d89d792 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:12.921598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b9a9bb22-efc5-4483-8af8-74444d89d792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:13.028493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 687/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:13.068836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb176468-b56a-44f9-8230-41a09b5f9fe6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:13.070409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fb176468-b56a-44f9-8230-41a09b5f9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:13.121398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:13.178493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb176468-b56a-44f9-8230-41a09b5f9fe6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:13.256295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:13.256925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:13.257293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:13.257489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:13.259206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:13.357612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:13.378591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:13.380637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde56bd-a94e-4275-8eb0-d750bb077b1b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:13.399780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 656/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:16.048817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6507d-aa8d-4667-8c3f-9761cad793ad None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:16.049428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6507d-aa8d-4667-8c3f-9761cad793ad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:16.049428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6507d-aa8d-4667-8c3f-9761cad793ad None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:16.050775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 688/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:16.056709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daaa54-24ba-408c-8c4b-a35fa0b38b1d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:16.057285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daaa54-24ba-408c-8c4b-a35fa0b38b1d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:16.057285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daaa54-24ba-408c-8c4b-a35fa0b38b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:16.135834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-35daaa54-24ba-408c-8c4b-a35fa0b38b1d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:16.137067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-35daaa54-24ba-408c-8c4b-a35fa0b38b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:16.248551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 657/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:16.285898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6507d-aa8d-4667-8c3f-9761cad793ad None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:16.287587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fbd6507d-aa8d-4667-8c3f-9761cad793ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:16.380517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:16.399045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6507d-aa8d-4667-8c3f-9761cad793ad None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:16.480181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:16.480954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:16.481241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:16.481519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:16.483204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:16.601407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:16.602880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dce43-8639-4f8b-b620-48fdedc2816d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:16.622630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 689/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:20.717725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaf8b6-8384-42ed-b436-2272d4a25f4b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:20.718358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaf8b6-8384-42ed-b436-2272d4a25f4b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:20.718358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaf8b6-8384-42ed-b436-2272d4a25f4b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:20.719677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 658/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:20.724906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f99421-8f03-4cc0-b290-a5dc8f0b05c9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:20.725378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f99421-8f03-4cc0-b290-a5dc8f0b05c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:20.725378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f99421-8f03-4cc0-b290-a5dc8f0b05c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:20.740627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:20.813842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-01f99421-8f03-4cc0-b290-a5dc8f0b05c9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:20.815407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-01f99421-8f03-4cc0-b290-a5dc8f0b05c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:20.941482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 690/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:20.980231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaf8b6-8384-42ed-b436-2272d4a25f4b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:20.981824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4aaf8b6-8384-42ed-b436-2272d4a25f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:21.044464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:21.095249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaf8b6-8384-42ed-b436-2272d4a25f4b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:21.175081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:21.175723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:21.176015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:21.176282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:21.178038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:21.290068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:21.293449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:21.295059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad512a-480e-4c0d-b98d-fba935316efd service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:21.317157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 659/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:26.241139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582437-fe79-4c26-a29c-3856d6b25334 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:26.242008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582437-fe79-4c26-a29c-3856d6b25334 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:26.242453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9582437-fe79-4c26-a29c-3856d6b25334 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:26.245127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 691/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:05:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:26.250465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823aeb-2646-43bf-9c26-fd424eefc3ce None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:26.250974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823aeb-2646-43bf-9c26-fd424eefc3ce None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:26.251071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823aeb-2646-43bf-9c26-fd424eefc3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:26.340511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3c823aeb-2646-43bf-9c26-fd424eefc3ce None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:26.341901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3c823aeb-2646-43bf-9c26-fd424eefc3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:26.528769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 660/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:05:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:30.804310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a905c-5d9c-4401-b420-285f7034cbad None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:30.805190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a905c-5d9c-4401-b420-285f7034cbad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:30.805648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a905c-5d9c-4401-b420-285f7034cbad None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:30.807341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 692/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:30.812757 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0f94-e6e4-4e90-9853-7f93fede8d8a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:30.813246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0f94-e6e4-4e90-9853-7f93fede8d8a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:30.813555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0f94-e6e4-4e90-9853-7f93fede8d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:30.871689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:30.898014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4d0c0f94-e6e4-4e90-9853-7f93fede8d8a None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:30.899275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4d0c0f94-e6e4-4e90-9853-7f93fede8d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:30.939440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=59 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:31.038627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 661/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:05:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:32.270032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fed7c1-981d-445c-9d72-a8f24f83d437 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:32.270578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fed7c1-981d-445c-9d72-a8f24f83d437 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:32.270578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fed7c1-981d-445c-9d72-a8f24f83d437 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:32.272241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 693/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:32.279267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0431f1f-c9bd-4aa3-8b65-2a3b0fd238af None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:32.279797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0431f1f-c9bd-4aa3-8b65-2a3b0fd238af None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:32.279921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0431f1f-c9bd-4aa3-8b65-2a3b0fd238af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:32.369516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e0431f1f-c9bd-4aa3-8b65-2a3b0fd238af None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:32.369516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e0431f1f-c9bd-4aa3-8b65-2a3b0fd238af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:32.413768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:32.486375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 662/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:05:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:35.984947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cec2e1-5e34-4898-aaa2-c7a20f2649e4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:35.985516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cec2e1-5e34-4898-aaa2-c7a20f2649e4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:35.985516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cec2e1-5e34-4898-aaa2-c7a20f2649e4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:35.986869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 694/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:35.993217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d915-a830-4099-9b3d-60c42df4ff92 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:35.993690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d915-a830-4099-9b3d-60c42df4ff92 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:35.993940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d915-a830-4099-9b3d-60c42df4ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:36.110047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0719d915-a830-4099-9b3d-60c42df4ff92 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:36.111952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0719d915-a830-4099-9b3d-60c42df4ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:36.291047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 663/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:36.360562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a928f-d15a-458a-9a0d-c7ec62ad7d51 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:36.361427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a928f-d15a-458a-9a0d-c7ec62ad7d51 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:36.362367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a928f-d15a-458a-9a0d-c7ec62ad7d51 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:36.365424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 695/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 01:05:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:36.378297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7e977-c7dc-41a3-ad8a-5a6ad8727449 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:36.379078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7e977-c7dc-41a3-ad8a-5a6ad8727449 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:36.380187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7e977-c7dc-41a3-ad8a-5a6ad8727449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:36.403105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:36.476125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b8a7e977-c7dc-41a3-ad8a-5a6ad8727449 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:36.478030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b8a7e977-c7dc-41a3-ad8a-5a6ad8727449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:36.661117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 664/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 01:05:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:36.672064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a928f-d15a-458a-9a0d-c7ec62ad7d51 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:36.673889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e69a928f-d15a-458a-9a0d-c7ec62ad7d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:36.856576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a928f-d15a-458a-9a0d-c7ec62ad7d51 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:36.943869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:36.992495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:36.994953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:36.994953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:36.994953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:36.995501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:37.094614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:37.096287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa517ef-9110-497d-8122-130c81c31c4f service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:37.116728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 696/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:05:39.400897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8c5a2-380b-493d-ac90-8059096226d6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:39.401437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8c5a2-380b-493d-ac90-8059096226d6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:39.401437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8c5a2-380b-493d-ac90-8059096226d6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:39.402911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 665/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:39.413008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7681ad-867f-42f4-9cda-002a9fc0814e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:39.413585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7681ad-867f-42f4-9cda-002a9fc0814e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:39.413585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7681ad-867f-42f4-9cda-002a9fc0814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:39.492608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0d7681ad-867f-42f4-9cda-002a9fc0814e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:39.494073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0d7681ad-867f-42f4-9cda-002a9fc0814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:39.537666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:39.618976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 697/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:39.660122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8c5a2-380b-493d-ac90-8059096226d6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:39.661626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1b8c5a2-380b-493d-ac90-8059096226d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:39.744592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:39.844645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8c5a2-380b-493d-ac90-8059096226d6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:40.016788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:40.017431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:40.017758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:40.017986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:40.019663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:40.046048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:40.120331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:40.122134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4d783-c094-4493-8de9-752fa632948e service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:40.143264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 666/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:42.530893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71172e-8609-4188-9ffe-e30cfdb55204 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:42.531783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71172e-8609-4188-9ffe-e30cfdb55204 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:42.532069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71172e-8609-4188-9ffe-e30cfdb55204 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:42.533464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 698/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:42.539112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c49959-87f5-4698-8772-1cfc3ee76a49 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:42.539967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c49959-87f5-4698-8772-1cfc3ee76a49 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:42.540304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c49959-87f5-4698-8772-1cfc3ee76a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:42.635347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-52c49959-87f5-4698-8772-1cfc3ee76a49 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:42.636940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52c49959-87f5-4698-8772-1cfc3ee76a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:42.746249 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:42.749771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 667/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:42.802271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398c399-8fb3-4280-ba7c-5152eb4e9e1b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:42.802876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398c399-8fb3-4280-ba7c-5152eb4e9e1b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:42.803307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398c399-8fb3-4280-ba7c-5152eb4e9e1b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:42.805103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 699/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 01:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:05:42.810991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df885ac-d14f-4f54-8a8f-e2d23055b8e1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:42.811466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df885ac-d14f-4f54-8a8f-e2d23055b8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:42.811965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df885ac-d14f-4f54-8a8f-e2d23055b8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:43.038758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3df885ac-d14f-4f54-8a8f-e2d23055b8e1 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:43.041045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3df885ac-d14f-4f54-8a8f-e2d23055b8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:43.192850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 668/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:43.200832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398c399-8fb3-4280-ba7c-5152eb4e9e1b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:43.202381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7398c399-8fb3-4280-ba7c-5152eb4e9e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:43.360912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398c399-8fb3-4280-ba7c-5152eb4e9e1b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:43.386897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:43.497368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:43.498296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:43.498744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:43.499186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:43.501009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:43.606451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:43.608298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c34819-6d5c-4fb7-83ad-63d4e404c0c8 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:43.630632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 700/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:44.473235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc607e0-e770-4373-8fed-d674ed602bde None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:44.474823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc607e0-e770-4373-8fed-d674ed602bde None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:44.475351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc607e0-e770-4373-8fed-d674ed602bde None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:44.477398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 669/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:44.483557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061299-c1ae-4e7c-a344-6e0920c22b53 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:44.483962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061299-c1ae-4e7c-a344-6e0920c22b53 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:44.484385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061299-c1ae-4e7c-a344-6e0920c22b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:44.520616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:44.559234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-63061299-c1ae-4e7c-a344-6e0920c22b53 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:44.560639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-63061299-c1ae-4e7c-a344-6e0920c22b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:44.667479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 701/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:44.980186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9eac9-57ab-4ece-80e1-3625d41f4316 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:44.980558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9eac9-57ab-4ece-80e1-3625d41f4316 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:44.980914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9eac9-57ab-4ece-80e1-3625d41f4316 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:44.982762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 670/1371] 10.222.0.17 () {54 vars in 964 bytes} [Thu Jul 28 01:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:44.988309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6cfd-4cdb-4ae9-b85f-488259d82415 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:44.988727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6cfd-4cdb-4ae9-b85f-488259d82415 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:44.989092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6cfd-4cdb-4ae9-b85f-488259d82415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:45.077329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b90f6cfd-4cdb-4ae9-b85f-488259d82415 None None] MFA Rules not processed for user `6146fcd207164df18b9c08e8fcc1adb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:45.078795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b90f6cfd-4cdb-4ae9-b85f-488259d82415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:45.141140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:45.176679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 702/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Jul 28 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:45.184624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9eac9-57ab-4ece-80e1-3625d41f4316 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:45.186419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a7b9eac9-57ab-4ece-80e1-3625d41f4316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:45.241550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:45.306287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9eac9-57ab-4ece-80e1-3625d41f4316 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:45.388118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:45.389038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:45.389415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:45.389776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:45.391564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:45.506750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:45.524733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:45.526192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a27e24-250b-475e-b67a-86160d413a12 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:45.545640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 671/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:45.743685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999355e-4dfb-41c4-89e1-99f73ff5ee47 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:45.744136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999355e-4dfb-41c4-89e1-99f73ff5ee47 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:45.744356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999355e-4dfb-41c4-89e1-99f73ff5ee47 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:45.746546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 703/1374] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:45.751466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6dd4d-2ac2-4214-b93a-e1445d9b2774 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:45.751466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6dd4d-2ac2-4214-b93a-e1445d9b2774 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:45.752093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6dd4d-2ac2-4214-b93a-e1445d9b2774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:45.829733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-efa6dd4d-2ac2-4214-b93a-e1445d9b2774 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:45.830979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-efa6dd4d-2ac2-4214-b93a-e1445d9b2774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:45.932220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 672/1375] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:45.976294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999355e-4dfb-41c4-89e1-99f73ff5ee47 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:45.977902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7999355e-4dfb-41c4-89e1-99f73ff5ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:46.137236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999355e-4dfb-41c4-89e1-99f73ff5ee47 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:46.191015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:46.257305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:46.257943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:46.258203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:46.258475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:46.260324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:46.355015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:46.356644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e744f73-c201-4f16-96c4-d367b571803b service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:46.380573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 704/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:48.446139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd69ae-8cdc-42d6-8621-b1579623cb14 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:48.447213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd69ae-8cdc-42d6-8621-b1579623cb14 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:48.447454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd69ae-8cdc-42d6-8621-b1579623cb14 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:48.449021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 673/1377] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:48.454592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc85ded-d2e9-4384-ac42-de8daa9db162 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:48.455068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc85ded-d2e9-4384-ac42-de8daa9db162 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:48.455201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc85ded-d2e9-4384-ac42-de8daa9db162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:48.534499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6fc85ded-d2e9-4384-ac42-de8daa9db162 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:48.535845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6fc85ded-d2e9-4384-ac42-de8daa9db162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:48.543212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:48.653673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 705/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:48.705325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd69ae-8cdc-42d6-8621-b1579623cb14 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:48.706776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-42dd69ae-8cdc-42d6-8621-b1579623cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:48.771158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:48.863524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd69ae-8cdc-42d6-8621-b1579623cb14 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:48.995680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:48.996263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:48.996516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:48.996793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:48.998880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:49.025282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:49.139895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:49.142573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e101f1-de58-4815-b2a6-58086a9c972d service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:49.172526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 674/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:51.281436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a3155-a3e5-4192-aae1-971091a2c611 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:51.282000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a3155-a3e5-4192-aae1-971091a2c611 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:51.282000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a3155-a3e5-4192-aae1-971091a2c611 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:51.283528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 706/1380] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:51.289477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0da56-665e-42b9-88d5-76d3e2e1522d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:51.290028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0da56-665e-42b9-88d5-76d3e2e1522d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:51.290028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0da56-665e-42b9-88d5-76d3e2e1522d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:51.367144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-45d0da56-665e-42b9-88d5-76d3e2e1522d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:51.368436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-45d0da56-665e-42b9-88d5-76d3e2e1522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:51.466495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:51.486974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 675/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:51.523652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a3155-a3e5-4192-aae1-971091a2c611 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:51.525234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-371a3155-a3e5-4192-aae1-971091a2c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:51.680702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:51.692675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a3155-a3e5-4192-aae1-971091a2c611 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:51.812296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:51.812813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:51.813120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:51.813400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:51.815057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:51.913236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:51.914866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066ccb7-0438-46bb-864e-f2b341559fcc service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:51.926517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:51.942157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 707/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:54.037753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f25c-e9b4-43ae-a1d9-bb725244479e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:54.038395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f25c-e9b4-43ae-a1d9-bb725244479e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:54.038395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f25c-e9b4-43ae-a1d9-bb725244479e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:54.039983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 676/1383] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:54.045493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dbf03-ca0a-499c-b819-ed212b78ffd1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:54.046162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dbf03-ca0a-499c-b819-ed212b78ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:54.046162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dbf03-ca0a-499c-b819-ed212b78ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:54.120045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-003dbf03-ca0a-499c-b819-ed212b78ffd1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:54.121716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-003dbf03-ca0a-499c-b819-ed212b78ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:54.229696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 708/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:54.278966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f25c-e9b4-43ae-a1d9-bb725244479e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:54.280703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0745f25c-e9b4-43ae-a1d9-bb725244479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:54.454403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f25c-e9b4-43ae-a1d9-bb725244479e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:54.529378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:54.578430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:54.579217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:54.579475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:54.579826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:54.581637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:54.681036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:54.682706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06725d-3336-4994-b32d-1a8aa244bc37 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:54.702100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 677/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:56.751438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1f7b7-a71a-43ef-8d0d-e81539901d85 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:56.751998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1f7b7-a71a-43ef-8d0d-e81539901d85 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:56.751998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1f7b7-a71a-43ef-8d0d-e81539901d85 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:56.753449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 709/1386] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:56.759682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a08d00-f7e0-441e-84df-15e5ae2150d7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:56.760178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a08d00-f7e0-441e-84df-15e5ae2150d7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:56.760482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a08d00-f7e0-441e-84df-15e5ae2150d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:56.848125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-16a08d00-f7e0-441e-84df-15e5ae2150d7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:56.849424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-16a08d00-f7e0-441e-84df-15e5ae2150d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:56.923398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:57.013417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 678/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:57.061477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1f7b7-a71a-43ef-8d0d-e81539901d85 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:57.063136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8da1f7b7-a71a-43ef-8d0d-e81539901d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:57.150417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:57.245056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1f7b7-a71a-43ef-8d0d-e81539901d85 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:05:57.363826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:05:57.364490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:57.364776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:57.365055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:57.366757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:57.408039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:05:57.462540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:05:57.464235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bde88-2361-4f73-a082-a73874cec772 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:05:57.487462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 710/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:05:59.612986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908f71b-d8a8-4515-91d0-a3d94728f9da None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:59.613714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908f71b-d8a8-4515-91d0-a3d94728f9da None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:59.613714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908f71b-d8a8-4515-91d0-a3d94728f9da None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:59.614928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 679/1389] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:05:59.620678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ac2dc-752a-4f0f-9569-45bb690e5f37 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:05:59.621185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ac2dc-752a-4f0f-9569-45bb690e5f37 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:05:59.621185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ac2dc-752a-4f0f-9569-45bb690e5f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:05:59.692502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e81ac2dc-752a-4f0f-9569-45bb690e5f37 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:05:59.693755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e81ac2dc-752a-4f0f-9569-45bb690e5f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:05:59.801689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 711/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:05:59.847506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908f71b-d8a8-4515-91d0-a3d94728f9da None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:05:59.849236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a908f71b-d8a8-4515-91d0-a3d94728f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:00.004446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908f71b-d8a8-4515-91d0-a3d94728f9da None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:00.021761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:00.123979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:00.124531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:00.124813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:00.125089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:00.126795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:00.218705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:00.220346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc9eb-25c4-4bcb-a9a3-2017c4913971 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:00.238610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 680/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:00.716894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c13967-e299-4b60-ae7b-848a9bd6a931 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:00.717492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c13967-e299-4b60-ae7b-848a9bd6a931 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:00.717492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c13967-e299-4b60-ae7b-848a9bd6a931 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:00.718884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 712/1392] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:00.724360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30beee-1ea7-41cf-a81c-86ad24f48eaf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:00.724815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30beee-1ea7-41cf-a81c-86ad24f48eaf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:00.724946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30beee-1ea7-41cf-a81c-86ad24f48eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:00.753369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:00.797964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4d30beee-1ea7-41cf-a81c-86ad24f48eaf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:00.799244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4d30beee-1ea7-41cf-a81c-86ad24f48eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:00.903120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 681/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:00.958503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558fa4b-5250-4c49-b466-791c63e56e67 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:00.958791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558fa4b-5250-4c49-b466-791c63e56e67 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:00.959104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558fa4b-5250-4c49-b466-791c63e56e67 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:00.960518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 713/1394] 10.222.0.17 () {54 vars in 966 bytes} [Thu Jul 28 01:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:00.966120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae216b-af12-4914-9bb2-ef6d3c6b0d10 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:00.966419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae216b-af12-4914-9bb2-ef6d3c6b0d10 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:00.966698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae216b-af12-4914-9bb2-ef6d3c6b0d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:01.048375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dcae216b-af12-4914-9bb2-ef6d3c6b0d10 None None] MFA Rules not processed for user `a0c7dc51159c4fcd8ae520f82ef7ef61`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:01.049519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dcae216b-af12-4914-9bb2-ef6d3c6b0d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:01.093169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:01.132479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 682/1395] 10.222.0.17 () {58 vars in 1063 bytes} [Thu Jul 28 01:06:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:01.140251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558fa4b-5250-4c49-b466-791c63e56e67 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:01.141793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b558fa4b-5250-4c49-b466-791c63e56e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:01.156260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:01.247738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558fa4b-5250-4c49-b466-791c63e56e67 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:01.322927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:01.323568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:01.323713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:01.324014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:01.325663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:01.411646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:01.455760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:01.457214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0574d85-a95a-48f0-b015-26bace5bfbc6 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:01.480272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 714/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:02.232253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9c29-79a5-499b-aa89-f7ded8ee3338 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.232818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9c29-79a5-499b-aa89-f7ded8ee3338 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.232913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9c29-79a5-499b-aa89-f7ded8ee3338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.278376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c7bbd-eb4a-453e-9e79-e809f0df3829 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.279011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c7bbd-eb4a-453e-9e79-e809f0df3829 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.279011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c7bbd-eb4a-453e-9e79-e809f0df3829 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.280359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 715/1397] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:02.285573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.285848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.286126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.306531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-734c9c29-79a5-499b-aa89-f7ded8ee3338 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:02.307830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-734c9c29-79a5-499b-aa89-f7ded8ee3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:02.358787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:02.360062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:02.405450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 683/1398] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:06:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:02.449865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d949f3-e2d3-4040-927b-d28f777aae7a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.450156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d949f3-e2d3-4040-927b-d28f777aae7a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.450443 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d949f3-e2d3-4040-927b-d28f777aae7a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.451875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 684/1399] 10.222.0.17 () {54 vars in 943 bytes} [Thu Jul 28 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:02.458155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.458435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.458735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.469156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 716/1400] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:02.514373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:02.515641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:02.534234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:02.543187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] MFA Rules not processed for user `b4423ec5dc2f465086495db19afde175`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:02.543606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:02.544556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:02.634913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc38dd-e250-4c19-b8ff-ee76de4267bb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:02.656167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 685/1401] 10.222.0.17 () {58 vars in 1040 bytes} [Thu Jul 28 01:06:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:02.665632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:02.674960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117fb89-2066-45e2-a9d3-37785296867e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:02.751906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:02.752541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.752824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.753098 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.755116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:02.772850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:02.784949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:02.785567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:02.785867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:02.786130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:02.787851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:02.790721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:02.845796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:02.847421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980984e-22d2-40c8-8485-7a488c47013e service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:02.866055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 717/1402] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:02.878120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:02.880176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650917c0-6178-45dc-88d3-2433d6806deb service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:02.888354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 686/1403] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:06:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:06:04.922440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae242117-ac08-4b4b-82a6-6b5408c6b01e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:04.923140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae242117-ac08-4b4b-82a6-6b5408c6b01e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:04.923140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae242117-ac08-4b4b-82a6-6b5408c6b01e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:04.924378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 718/1404] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:04.929645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19746f-4d11-4efb-8b04-adcc6f82643e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:04.930574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19746f-4d11-4efb-8b04-adcc6f82643e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:04.931111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19746f-4d11-4efb-8b04-adcc6f82643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:05.008446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3d19746f-4d11-4efb-8b04-adcc6f82643e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:05.010113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3d19746f-4d11-4efb-8b04-adcc6f82643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:05.115516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:05.126942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 687/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:05.171152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae242117-ac08-4b4b-82a6-6b5408c6b01e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:05.172868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ae242117-ac08-4b4b-82a6-6b5408c6b01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:05.342027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:05.344727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae242117-ac08-4b4b-82a6-6b5408c6b01e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:05.513911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:05.514564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:05.514886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:05.515173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:05.520932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:05.646653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:05.647337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ac255-5d10-4b88-959a-38fc92ce5bce service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:05.673361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 719/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:07.778163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbab66e-7348-4364-8f0d-4a5b1bec26a6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:07.778893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbab66e-7348-4364-8f0d-4a5b1bec26a6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:07.779015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbab66e-7348-4364-8f0d-4a5b1bec26a6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:07.780625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 688/1407] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:07.785805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cbf03-7aa7-4e18-afb4-749a0079f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:07.786285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cbf03-7aa7-4e18-afb4-749a0079f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:07.786376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cbf03-7aa7-4e18-afb4-749a0079f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:07.791615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:07.879367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-032cbf03-7aa7-4e18-afb4-749a0079f7d2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:07.881011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-032cbf03-7aa7-4e18-afb4-749a0079f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:07.995790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 720/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:08.041082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbab66e-7348-4364-8f0d-4a5b1bec26a6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:08.042641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dbbab66e-7348-4364-8f0d-4a5b1bec26a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:08.219211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbab66e-7348-4364-8f0d-4a5b1bec26a6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:08.305022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:08.365555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:08.366338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:08.366829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:08.367350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:08.369407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:08.482578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:08.484716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bc482-0beb-43a4-b6e6-32ebe53f4d38 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:08.509797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 689/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:17.993755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=64 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:18.506794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=64 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:26.608276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e43ef-797d-4f31-8079-6c7184d43b19 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:26.608945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e43ef-797d-4f31-8079-6c7184d43b19 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:26.608945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e43ef-797d-4f31-8079-6c7184d43b19 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:26.610333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 721/1410] 10.222.0.17 () {54 vars in 918 bytes} [Thu Jul 28 01:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:06:26.616923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0be23-a988-4789-80af-a35520655a88 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:26.617861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0be23-a988-4789-80af-a35520655a88 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:26.618039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0be23-a988-4789-80af-a35520655a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:26.714647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d8a0be23-a988-4789-80af-a35520655a88 None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:26.715960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d8a0be23-a988-4789-80af-a35520655a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:26.842539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 690/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jul 28 01:06:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:26.859758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b7893-57bb-40a7-9f4c-289558d8e8c2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:26.860102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b7893-57bb-40a7-9f4c-289558d8e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:26.860310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b7893-57bb-40a7-9f4c-289558d8e8c2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:26.861754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 722/1412] 10.222.0.17 () {54 vars in 918 bytes} [Thu Jul 28 01:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:26.868806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec00435-22ed-45a9-af29-ff2964dfa622 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:26.869307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec00435-22ed-45a9-af29-ff2964dfa622 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:26.869776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec00435-22ed-45a9-af29-ff2964dfa622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:26.971947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:26.975348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3ec00435-22ed-45a9-af29-ff2964dfa622 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:26.976764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3ec00435-22ed-45a9-af29-ff2964dfa622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:27.166791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 691/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jul 28 01:06:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:29.944140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306362f5-a1ce-422c-8f03-ebc7f356c122 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:29.944865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306362f5-a1ce-422c-8f03-ebc7f356c122 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:29.944865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306362f5-a1ce-422c-8f03-ebc7f356c122 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:29.946379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 723/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:29.955543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073e045-3ab4-4d29-8989-14bb430447d2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:29.956213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073e045-3ab4-4d29-8989-14bb430447d2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:29.956213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073e045-3ab4-4d29-8989-14bb430447d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:30.039656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c073e045-3ab4-4d29-8989-14bb430447d2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:30.041115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c073e045-3ab4-4d29-8989-14bb430447d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:30.111659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:30.163742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 692/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:30.217934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306362f5-a1ce-422c-8f03-ebc7f356c122 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:30.220885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-306362f5-a1ce-422c-8f03-ebc7f356c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:30.413088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306362f5-a1ce-422c-8f03-ebc7f356c122 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:30.500278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:30.554425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:30.555130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:30.555428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:30.555709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:30.557473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:30.662855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:30.664660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ae8b9-da35-41d8-bf15-3e075d8f94a0 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:30.686733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 724/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:31.653520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966ab36-f9a2-4a44-b698-9f094389b609 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:31.654130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966ab36-f9a2-4a44-b698-9f094389b609 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:31.654130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966ab36-f9a2-4a44-b698-9f094389b609 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:31.655498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 693/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:31.661718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f94002-aaf3-4a31-bd10-a5a141cde073 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:31.662371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f94002-aaf3-4a31-bd10-a5a141cde073 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:31.662371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f94002-aaf3-4a31-bd10-a5a141cde073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:31.756299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-67f94002-aaf3-4a31-bd10-a5a141cde073 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:31.757511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-67f94002-aaf3-4a31-bd10-a5a141cde073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:31.791878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:31.879028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 725/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:32.033179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966ab36-f9a2-4a44-b698-9f094389b609 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:32.034729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0966ab36-f9a2-4a44-b698-9f094389b609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:32.259297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966ab36-f9a2-4a44-b698-9f094389b609 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:32.312302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:32.411850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:32.412516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:32.412796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:32.413066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:32.416150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:32.537826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:32.539640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400dd4b-d6c8-4368-9470-0943c4a256e3 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:32.563387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 694/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:33.517548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55301a58-637f-42f9-88e0-641e2f18c008 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:33.518412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55301a58-637f-42f9-88e0-641e2f18c008 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:33.518412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55301a58-637f-42f9-88e0-641e2f18c008 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:33.519812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 726/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:33.526566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0af00-929f-454f-abff-35a019ee447a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:33.527432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0af00-929f-454f-abff-35a019ee447a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:33.527432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0af00-929f-454f-abff-35a019ee447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:33.595013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:33.623071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8fc0af00-929f-454f-abff-35a019ee447a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:33.624620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8fc0af00-929f-454f-abff-35a019ee447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:33.749597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 695/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:33.907830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55301a58-637f-42f9-88e0-641e2f18c008 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:33.909810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-55301a58-637f-42f9-88e0-641e2f18c008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:34.092527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55301a58-637f-42f9-88e0-641e2f18c008 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:34.108409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:34.235552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:34.236210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:34.236492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:34.236792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:34.238618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:34.351731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:34.353391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce15d9-afd3-4d32-9759-313a588929d0 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:34.376002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 727/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:06:35.343511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33f57a-687e-43fd-8eec-969fe4c2801c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:35.344124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33f57a-687e-43fd-8eec-969fe4c2801c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:35.344124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33f57a-687e-43fd-8eec-969fe4c2801c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:35.345497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 696/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:06:35.351832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8586ce-d887-4e55-afe3-962c78fa43e1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:35.352589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8586ce-d887-4e55-afe3-962c78fa43e1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:35.352589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8586ce-d887-4e55-afe3-962c78fa43e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:35.376171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:35.450595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6e8586ce-d887-4e55-afe3-962c78fa43e1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:35.451928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6e8586ce-d887-4e55-afe3-962c78fa43e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:35.579078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 728/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:35.747627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33f57a-687e-43fd-8eec-969fe4c2801c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:35.749316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ed33f57a-687e-43fd-8eec-969fe4c2801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:35.886868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:35.931386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33f57a-687e-43fd-8eec-969fe4c2801c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:36.079260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:36.079824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:36.080110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:36.080382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:36.082158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:36.173212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:36.190153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:36.192173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168dcdc-024c-48f2-99ce-f1d18b7da91b service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:36.220021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 697/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:37.136553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0ab15-d4a6-4d80-a5a9-8dbb77112a17 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:37.137094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0ab15-d4a6-4d80-a5a9-8dbb77112a17 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:37.137094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0ab15-d4a6-4d80-a5a9-8dbb77112a17 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:37.138409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 729/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:37.143965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c245-16c7-4bfc-9039-c80e2d209bdb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:37.144549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c245-16c7-4bfc-9039-c80e2d209bdb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:37.144549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c245-16c7-4bfc-9039-c80e2d209bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:37.240532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4623c245-16c7-4bfc-9039-c80e2d209bdb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:37.241878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4623c245-16c7-4bfc-9039-c80e2d209bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:37.361235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 698/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:37.515795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0ab15-d4a6-4d80-a5a9-8dbb77112a17 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:37.517376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-36b0ab15-d4a6-4d80-a5a9-8dbb77112a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:37.620567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:37.725557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0ab15-d4a6-4d80-a5a9-8dbb77112a17 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:37.873210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:37.873945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:37.874269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:37.874665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:37.876812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:37.935189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:37.996870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:37.999908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e9181-f542-452a-aa71-c603efe5b42e service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:38.026297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 730/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:06:38.890610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855a5a-7432-43c3-9be8-374b8a09a692 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:38.891333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855a5a-7432-43c3-9be8-374b8a09a692 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:38.891333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855a5a-7432-43c3-9be8-374b8a09a692 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:38.892650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 699/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:38.898620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb364f2a-a9b4-4464-92de-f03943af077e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:38.899287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb364f2a-a9b4-4464-92de-f03943af077e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:38.899287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb364f2a-a9b4-4464-92de-f03943af077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:38.991071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fb364f2a-a9b4-4464-92de-f03943af077e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:38.992461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fb364f2a-a9b4-4464-92de-f03943af077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:39.111657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 731/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:39.263442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855a5a-7432-43c3-9be8-374b8a09a692 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:39.265071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5f855a5a-7432-43c3-9be8-374b8a09a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:39.326995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:39.444066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855a5a-7432-43c3-9be8-374b8a09a692 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:39.591263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:39.592063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:39.592439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:39.592801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:39.594717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:39.604036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:39.713895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:39.715818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1f6df-f13d-4481-850e-61e4dd2f9920 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:39.739121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 700/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:06:40.599545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4820254-fd8f-4409-aabd-c6ee055553d8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:40.600182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4820254-fd8f-4409-aabd-c6ee055553d8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:40.600332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4820254-fd8f-4409-aabd-c6ee055553d8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:40.601906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 732/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:40.608239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a31605-3873-4c98-98fa-e1a243653d9b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:40.608888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a31605-3873-4c98-98fa-e1a243653d9b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:40.608888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a31605-3873-4c98-98fa-e1a243653d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:40.701798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-37a31605-3873-4c98-98fa-e1a243653d9b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:40.703093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-37a31605-3873-4c98-98fa-e1a243653d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:40.782468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:40.820155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 701/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:06:41.090585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4820254-fd8f-4409-aabd-c6ee055553d8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:41.092501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d4820254-fd8f-4409-aabd-c6ee055553d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:41.114257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:41.311862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4820254-fd8f-4409-aabd-c6ee055553d8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:41.424527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:41.458967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:41.459625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:41.459910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:41.460187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:41.462116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:41.576740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:41.578815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc87a60-a8a6-4b30-b66b-82533a17145d service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:41.601294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 733/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:42.708112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ed51-43be-4334-84c8-7a2c5efe3d8d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:42.708695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ed51-43be-4334-84c8-7a2c5efe3d8d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:42.708871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ed51-43be-4334-84c8-7a2c5efe3d8d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:42.710378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 702/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:42.716669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117fcaf-529f-4ebd-bc3c-b45034e24ed6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:42.717555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117fcaf-529f-4ebd-bc3c-b45034e24ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:42.717555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117fcaf-529f-4ebd-bc3c-b45034e24ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:42.810289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f117fcaf-529f-4ebd-bc3c-b45034e24ed6 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:42.811587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f117fcaf-529f-4ebd-bc3c-b45034e24ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:42.869874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:42.974810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 734/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:43.128577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7ed51-43be-4334-84c8-7a2c5efe3d8d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:43.130854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9aa7ed51-43be-4334-84c8-7a2c5efe3d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:43.344570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7ed51-43be-4334-84c8-7a2c5efe3d8d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:43.418867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:43.497204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:43.498000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:43.498379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:43.498753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:43.501086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:43.610052 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:43.611715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ab02a-f249-44c5-abe8-e42b77e5837e service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:43.633599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 703/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:06:44.497893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533fbf8-facf-42e5-8513-7c7bf918ef54 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:44.498617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533fbf8-facf-42e5-8513-7c7bf918ef54 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:44.498617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533fbf8-facf-42e5-8513-7c7bf918ef54 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:44.499902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 735/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:44.506618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e847bcf-9bbf-4ff2-8f05-c7c276f924ec None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:44.507275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e847bcf-9bbf-4ff2-8f05-c7c276f924ec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:44.507453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e847bcf-9bbf-4ff2-8f05-c7c276f924ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:44.609592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6e847bcf-9bbf-4ff2-8f05-c7c276f924ec None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:44.611132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6e847bcf-9bbf-4ff2-8f05-c7c276f924ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:44.613745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:44.738306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 704/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:44.899325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533fbf8-facf-42e5-8513-7c7bf918ef54 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:44.901039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e533fbf8-facf-42e5-8513-7c7bf918ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:45.090937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533fbf8-facf-42e5-8513-7c7bf918ef54 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:45.126913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:45.235709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:45.236433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:45.236934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:45.237256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:45.239699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:45.367548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:45.369307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c99c0-bc10-47f8-afd3-3f5e00c8c099 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:45.393346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 736/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:47.441614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a75d1-827e-47fc-9b3e-3a3ba05adfc3 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:47.442320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a75d1-827e-47fc-9b3e-3a3ba05adfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:47.442586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a75d1-827e-47fc-9b3e-3a3ba05adfc3 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:47.445230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 705/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:06:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:06:47.453119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56c4c4-4634-439e-8478-9b9b592dfc5f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:47.454033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56c4c4-4634-439e-8478-9b9b592dfc5f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:47.454033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56c4c4-4634-439e-8478-9b9b592dfc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:47.505137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:47.541291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-eb56c4c4-4634-439e-8478-9b9b592dfc5f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:47.542572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-eb56c4c4-4634-439e-8478-9b9b592dfc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:47.658967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 737/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:47.723319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a75d1-827e-47fc-9b3e-3a3ba05adfc3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:47.724959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-323a75d1-827e-47fc-9b3e-3a3ba05adfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:47.856197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a75d1-827e-47fc-9b3e-3a3ba05adfc3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:47.918185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:48.008360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:48.009429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:48.009916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:48.010431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:48.014492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:48.190945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:48.193827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f566b-3b63-4780-ae9c-df398633e1f0 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:48.221351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 706/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:06:50.697863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cffb6-c85b-48ef-b8cf-dd02ded70937 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:50.698681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cffb6-c85b-48ef-b8cf-dd02ded70937 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:50.698681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cffb6-c85b-48ef-b8cf-dd02ded70937 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:50.699914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 738/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:06:50.706607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dc8a6-63b0-4172-8fea-c4f2092aec4a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:50.707515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dc8a6-63b0-4172-8fea-c4f2092aec4a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:50.707515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dc8a6-63b0-4172-8fea-c4f2092aec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:50.798361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0b7dc8a6-63b0-4172-8fea-c4f2092aec4a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:50.799850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0b7dc8a6-63b0-4172-8fea-c4f2092aec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:50.815978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:50.935231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 707/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:50.979079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cffb6-c85b-48ef-b8cf-dd02ded70937 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:50.981021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a5cffb6-c85b-48ef-b8cf-dd02ded70937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:51.118020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cffb6-c85b-48ef-b8cf-dd02ded70937 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:51.134616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:51.217140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:51.217993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:51.218369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:51.218771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:51.220702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:51.362400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:51.364321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b385f49-280e-4e8b-aec4-e1deb3df0a2e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:51.390059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 739/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:54.008850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82c63a-94c6-4517-b189-8a00ff21e3c6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:54.009585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82c63a-94c6-4517-b189-8a00ff21e3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:54.009585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82c63a-94c6-4517-b189-8a00ff21e3c6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:54.010896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 708/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:06:54.017347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ba35d-4d9e-409b-843d-c21301809cbd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:54.017967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ba35d-4d9e-409b-843d-c21301809cbd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:54.017967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ba35d-4d9e-409b-843d-c21301809cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:54.077681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:54.100948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-813ba35d-4d9e-409b-843d-c21301809cbd None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:54.102291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-813ba35d-4d9e-409b-843d-c21301809cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:54.246740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 740/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:06:54.289111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82c63a-94c6-4517-b189-8a00ff21e3c6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:54.292162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac82c63a-94c6-4517-b189-8a00ff21e3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:54.427411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82c63a-94c6-4517-b189-8a00ff21e3c6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:54.483014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:54.526807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:54.527595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:54.527983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:54.528430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:54.530330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:54.673994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:54.676205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb8975-ce14-492c-9857-cc14967ace73 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:54.700978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 709/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:06:57.012336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485ba2-ba08-4bd1-aa9f-fc6c32c76d4f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:57.013057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485ba2-ba08-4bd1-aa9f-fc6c32c76d4f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:57.013057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485ba2-ba08-4bd1-aa9f-fc6c32c76d4f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:57.014544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 741/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:06:57.021270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddadea2-4156-4ded-a280-8b8f88b86400 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:57.022151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddadea2-4156-4ded-a280-8b8f88b86400 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:57.022151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddadea2-4156-4ded-a280-8b8f88b86400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:57.104906 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0ddadea2-4156-4ded-a280-8b8f88b86400 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:06:57.106219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ddadea2-4156-4ded-a280-8b8f88b86400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:57.131371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:57.232745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 710/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:06:57.279259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f485ba2-ba08-4bd1-aa9f-fc6c32c76d4f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:06:57.280921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9f485ba2-ba08-4bd1-aa9f-fc6c32c76d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:57.421192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f485ba2-ba08-4bd1-aa9f-fc6c32c76d4f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:06:57.449589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:06:57.521162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:06:57.521841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:57.522122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:57.522400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:57.524525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:06:57.666692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:06:57.668483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95452243-ce66-4980-877a-eb5347807959 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:06:57.692700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 742/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:06:59.950167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34172a7-3c70-439f-9b9d-11b28d235cc7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:59.951232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34172a7-3c70-439f-9b9d-11b28d235cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:59.951232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34172a7-3c70-439f-9b9d-11b28d235cc7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:06:59.952220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 711/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:06:59.958374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63a04b-5999-44a2-80b9-b216ce62c802 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:06:59.959049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63a04b-5999-44a2-80b9-b216ce62c802 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:06:59.959049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63a04b-5999-44a2-80b9-b216ce62c802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:00.038819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:00.042862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9b63a04b-5999-44a2-80b9-b216ce62c802 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:00.044479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9b63a04b-5999-44a2-80b9-b216ce62c802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:00.162160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 743/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:00.204969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34172a7-3c70-439f-9b9d-11b28d235cc7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:00.206516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d34172a7-3c70-439f-9b9d-11b28d235cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:00.334652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34172a7-3c70-439f-9b9d-11b28d235cc7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:00.404688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:00.441125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:00.441814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:00.442103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:00.442451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:00.444229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:00.587699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:00.589988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf52a5-399e-4d5c-9341-ca50a1e92939 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:00.615387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 712/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:07:02.934106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550afba0-a47c-4bf3-95ec-dfbf6c67fce1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:02.934747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550afba0-a47c-4bf3-95ec-dfbf6c67fce1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:02.934747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550afba0-a47c-4bf3-95ec-dfbf6c67fce1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:02.936399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 744/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:02.942816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023881-f8b0-4c8e-918b-8212135da651 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:02.943508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023881-f8b0-4c8e-918b-8212135da651 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:02.943508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023881-f8b0-4c8e-918b-8212135da651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:03.042107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6f023881-f8b0-4c8e-918b-8212135da651 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:03.043353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6f023881-f8b0-4c8e-918b-8212135da651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:03.079122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:03.164749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 713/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:03.203869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550afba0-a47c-4bf3-95ec-dfbf6c67fce1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:03.205706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-550afba0-a47c-4bf3-95ec-dfbf6c67fce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:03.333893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550afba0-a47c-4bf3-95ec-dfbf6c67fce1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:03.367604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:03.423522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:03.424160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:03.424446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:03.424727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:03.426487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:03.560595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:03.562068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217c79-e079-4ab2-b01c-fc4afdf57efe service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:03.583297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 745/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:05.862661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7fb2c-d586-4b85-8682-43914e5a7ca6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:05.863321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7fb2c-d586-4b85-8682-43914e5a7ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:05.863321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7fb2c-d586-4b85-8682-43914e5a7ca6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:05.865214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 714/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:05.872210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971d64-0315-46fa-9bdf-6404b0d1a947 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:05.872863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971d64-0315-46fa-9bdf-6404b0d1a947 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:05.872863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971d64-0315-46fa-9bdf-6404b0d1a947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:05.991006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4e971d64-0315-46fa-9bdf-6404b0d1a947 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:05.993076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4e971d64-0315-46fa-9bdf-6404b0d1a947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:06.009236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:06.122895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 746/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:07:06.162518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7fb2c-d586-4b85-8682-43914e5a7ca6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:06.164559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6eb7fb2c-d586-4b85-8682-43914e5a7ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:06.302997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7fb2c-d586-4b85-8682-43914e5a7ca6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:06.372232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:06.396611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:06.397187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:06.397474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:06.397971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:06.399854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:06.560160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:06.561889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06cb16-1586-4fac-aa8c-8452a76e6587 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:06.590725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 715/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:08.792711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e674a-c7b2-43da-acba-3cbcfb3aeb4b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:08.793375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e674a-c7b2-43da-acba-3cbcfb3aeb4b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:08.793375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e674a-c7b2-43da-acba-3cbcfb3aeb4b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:08.794888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 747/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:08.801218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e304e50-9710-4cbe-ab32-69d86b7471b8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:08.801931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e304e50-9710-4cbe-ab32-69d86b7471b8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:08.801931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e304e50-9710-4cbe-ab32-69d86b7471b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:08.889044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1e304e50-9710-4cbe-ab32-69d86b7471b8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:08.890315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1e304e50-9710-4cbe-ab32-69d86b7471b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:08.936429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:09.022990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 716/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:09.063496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e674a-c7b2-43da-acba-3cbcfb3aeb4b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:09.065083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-813e674a-c7b2-43da-acba-3cbcfb3aeb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:09.187749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e674a-c7b2-43da-acba-3cbcfb3aeb4b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:09.232294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:09.302944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:09.303609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:09.303898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:09.304216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:09.306164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:09.446305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:09.447857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1e9d4-ca9c-4293-97a7-4bc24faf205f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:09.470322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 748/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:11.635265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90952-bd06-45a7-87dc-8d588306b8ca None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:11.636129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90952-bd06-45a7-87dc-8d588306b8ca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:11.636129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90952-bd06-45a7-87dc-8d588306b8ca None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:11.637827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 717/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:11.644575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f694ff-924a-4a47-a512-09588c13f006 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:11.645202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f694ff-924a-4a47-a512-09588c13f006 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:11.645202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f694ff-924a-4a47-a512-09588c13f006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:11.720816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f0f694ff-924a-4a47-a512-09588c13f006 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:11.722227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f0f694ff-924a-4a47-a512-09588c13f006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:11.742898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:11.839415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 749/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:11.882785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90952-bd06-45a7-87dc-8d588306b8ca None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:11.884385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-73a90952-bd06-45a7-87dc-8d588306b8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:12.002715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90952-bd06-45a7-87dc-8d588306b8ca None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:12.078425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:12.086984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:12.087631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:12.087909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:12.088181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:12.089992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:12.223869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:12.225521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e83eb2-33a5-4b74-9d1e-f280b3c1467b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:12.247595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 718/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:07:14.370578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a676-8b35-4564-9717-503e669b33c1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:14.371343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a676-8b35-4564-9717-503e669b33c1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:14.371343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80a676-8b35-4564-9717-503e669b33c1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:14.372559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 750/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:14.379232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef849-576d-48e0-9583-1d93d34f671f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:14.380171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef849-576d-48e0-9583-1d93d34f671f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:14.380506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef849-576d-48e0-9583-1d93d34f671f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:14.495806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b88ef849-576d-48e0-9583-1d93d34f671f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:14.497128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b88ef849-576d-48e0-9583-1d93d34f671f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:14.558059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:14.644917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 719/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:14.681991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80a676-8b35-4564-9717-503e669b33c1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:14.683741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0f80a676-8b35-4564-9717-503e669b33c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:14.811297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80a676-8b35-4564-9717-503e669b33c1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:14.894333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:14.945483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:14.946597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:14.947312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:14.947843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:14.951231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:15.100602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:15.102186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa8e73-506a-4594-90ef-e66589a562dc service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:15.126778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 751/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:24.642525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=22 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:25.123718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=66 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:36.018153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e51b1-adee-44f1-9a5b-c110461e09d7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:36.018763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e51b1-adee-44f1-9a5b-c110461e09d7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:36.018763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e51b1-adee-44f1-9a5b-c110461e09d7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:36.020202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 720/1471] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:36.027217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9983be-fada-4a11-9d68-8ebc4da49f1c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:36.028122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9983be-fada-4a11-9d68-8ebc4da49f1c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:36.028122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9983be-fada-4a11-9d68-8ebc4da49f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:36.109424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1a9983be-fada-4a11-9d68-8ebc4da49f1c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:36.110665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a9983be-fada-4a11-9d68-8ebc4da49f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:36.226374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 752/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:07:36.330662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a96ae-4c5c-42b1-8f8e-3810c55afd11 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:36.331005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a96ae-4c5c-42b1-8f8e-3810c55afd11 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:36.331256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a96ae-4c5c-42b1-8f8e-3810c55afd11 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:36.332904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 721/1473] 10.222.0.17 () {54 vars in 966 bytes} [Thu Jul 28 01:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:07:36.339233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d7690-fa87-41a8-ab65-f905d839f6fd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:36.339557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d7690-fa87-41a8-ab65-f905d839f6fd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:36.339904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d7690-fa87-41a8-ab65-f905d839f6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:36.429967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-320d7690-fa87-41a8-ab65-f905d839f6fd None None] MFA Rules not processed for user `a0c7dc51159c4fcd8ae520f82ef7ef61`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:36.431263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-320d7690-fa87-41a8-ab65-f905d839f6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:36.456957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:36.536720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 753/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Thu Jul 28 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:36.544770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a96ae-4c5c-42b1-8f8e-3810c55afd11 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:36.546344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0d3a96ae-4c5c-42b1-8f8e-3810c55afd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:36.672253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a96ae-4c5c-42b1-8f8e-3810c55afd11 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:36.766776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:36.767447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:36.767726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:36.767998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:36.769792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:36.836528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:36.906659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:36.908195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd296cd-4ec2-41fb-9edc-1b53ea6e87c4 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:36.931616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 722/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:39.192347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c710e48-0d74-44a6-9769-0c98b1d4ec32 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:39.193379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c710e48-0d74-44a6-9769-0c98b1d4ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:39.193379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c710e48-0d74-44a6-9769-0c98b1d4ec32 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:39.194958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 754/1476] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:39.202331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e4b76-4a93-4813-af03-ac1821589fc5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:39.203015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e4b76-4a93-4813-af03-ac1821589fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:39.203015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e4b76-4a93-4813-af03-ac1821589fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:39.289123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-500e4b76-4a93-4813-af03-ac1821589fc5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:39.290412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-500e4b76-4a93-4813-af03-ac1821589fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:39.427583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 723/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:39.470733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c710e48-0d74-44a6-9769-0c98b1d4ec32 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:39.473950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6c710e48-0d74-44a6-9769-0c98b1d4ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:39.659592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c710e48-0d74-44a6-9769-0c98b1d4ec32 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:39.732057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:39.799402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:39.800174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:39.800458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:39.800731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:39.801464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:39.803347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:07:39.803347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:07:39.803347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:07:39.804521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1423c-1b30-4e9b-9e4a-463cb30a3a62 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:39.843508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 755/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jul 28 01:07:39 2022] GET /identity/v3/projects => generated 2553 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:07:41.963749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94de85a-2db8-451a-a191-9b6ca0332d3c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:41.964404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94de85a-2db8-451a-a191-9b6ca0332d3c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:41.964404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94de85a-2db8-451a-a191-9b6ca0332d3c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:41.965691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 724/1479] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:07:41.972257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81b712-de92-4c0f-b4ac-b114c73b2e3d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:41.973138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81b712-de92-4c0f-b4ac-b114c73b2e3d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:41.973138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81b712-de92-4c0f-b4ac-b114c73b2e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:42.058610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ba81b712-de92-4c0f-b4ac-b114c73b2e3d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:42.059906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ba81b712-de92-4c0f-b4ac-b114c73b2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:42.184306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 756/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:42.231265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94de85a-2db8-451a-a191-9b6ca0332d3c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:42.232971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a94de85a-2db8-451a-a191-9b6ca0332d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:42.244084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:42.406731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94de85a-2db8-451a-a191-9b6ca0332d3c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:42.509238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:42.553610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:42.554240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:42.554520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:42.554881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:42.556707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:42.676321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:42.678436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77768-74c5-4928-a158-2c9f1d5c4a6b service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:42.701184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 725/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:45.092434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae245dc7-257c-484a-ac03-98f38e2e54e9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:45.093283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae245dc7-257c-484a-ac03-98f38e2e54e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:45.093283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae245dc7-257c-484a-ac03-98f38e2e54e9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:45.095059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 757/1482] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:45.101958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c3a6-617b-492c-b84d-8d55983dc934 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:45.102570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c3a6-617b-492c-b84d-8d55983dc934 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:45.102570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c3a6-617b-492c-b84d-8d55983dc934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:45.188225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dd23c3a6-617b-492c-b84d-8d55983dc934 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:45.189680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dd23c3a6-617b-492c-b84d-8d55983dc934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:45.255573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:45.329951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 726/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:45.368547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae245dc7-257c-484a-ac03-98f38e2e54e9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:45.370884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ae245dc7-257c-484a-ac03-98f38e2e54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:45.401880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:45.507950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae245dc7-257c-484a-ac03-98f38e2e54e9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:45.596060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:45.596780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:45.597113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:45.597435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:45.599344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:45.693997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:45.737382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:45.738967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee33e8-8971-45aa-8f7f-d3034e9d2f59 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:45.762234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 758/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:07:47.964800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59ab2b-ddf1-4a1a-b519-1ae445a26b05 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:47.965681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59ab2b-ddf1-4a1a-b519-1ae445a26b05 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:47.965681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59ab2b-ddf1-4a1a-b519-1ae445a26b05 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:47.967568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 727/1485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:47.974601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53630e-1909-4ff5-af07-8fe3201801e8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:47.975686 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53630e-1909-4ff5-af07-8fe3201801e8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:47.975686 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53630e-1909-4ff5-af07-8fe3201801e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:48.057858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bf53630e-1909-4ff5-af07-8fe3201801e8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:48.059320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bf53630e-1909-4ff5-af07-8fe3201801e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:48.190784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 759/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:48.231438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59ab2b-ddf1-4a1a-b519-1ae445a26b05 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:48.234438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1d59ab2b-ddf1-4a1a-b519-1ae445a26b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:48.358508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59ab2b-ddf1-4a1a-b519-1ae445a26b05 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:48.455127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:48.457675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:48.458583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:48.459242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:48.459806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:48.462174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:48.609842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:48.611714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2004c0b-d27f-4008-a84c-8a0edfbb5333 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:48.635591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 728/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:51.009175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5faea-4873-4e43-aa9f-c1922f6b3905 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:51.009814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5faea-4873-4e43-aa9f-c1922f6b3905 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:51.009814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5faea-4873-4e43-aa9f-c1922f6b3905 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:51.011441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 760/1488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:51.017921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159aff2-a915-41c5-a62b-754730862189 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:51.018589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159aff2-a915-41c5-a62b-754730862189 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:51.018589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159aff2-a915-41c5-a62b-754730862189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:51.102237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c159aff2-a915-41c5-a62b-754730862189 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:51.103747 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c159aff2-a915-41c5-a62b-754730862189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:51.168593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:51.226221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 729/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:07:51.272011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5faea-4873-4e43-aa9f-c1922f6b3905 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:51.274047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8dd5faea-4873-4e43-aa9f-c1922f6b3905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:51.325440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:51.417521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5faea-4873-4e43-aa9f-c1922f6b3905 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:51.525560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:51.526202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:51.526500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:51.526779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:51.528567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:51.628714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:51.690461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:51.692763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791b1cf-1576-4937-acf8-b1ca848c6e58 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:51.738529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 761/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:07:54.193776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56456c8-bfc0-4be2-a728-7742cd36ca3a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:54.194383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56456c8-bfc0-4be2-a728-7742cd36ca3a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:54.194383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56456c8-bfc0-4be2-a728-7742cd36ca3a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:54.195864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 730/1491] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jul 28 01:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:07:54.202455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b03e-7209-48b9-aa48-6df361629901 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:54.203384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b03e-7209-48b9-aa48-6df361629901 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:54.203384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b03e-7209-48b9-aa48-6df361629901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:54.288317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-68e2b03e-7209-48b9-aa48-6df361629901 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:07:54.289598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-68e2b03e-7209-48b9-aa48-6df361629901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:54.411511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 762/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jul 28 01:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:07:54.474857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56456c8-bfc0-4be2-a728-7742cd36ca3a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:07:54.476548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b56456c8-bfc0-4be2-a728-7742cd36ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:54.623311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56456c8-bfc0-4be2-a728-7742cd36ca3a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:07:54.739404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:07:54.740417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:07:54.740919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:07:54.741423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:07:54.744736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:07:54.762978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:07:54.891929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:07:54.893500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5ab80-6c49-46e6-a51b-85c52ca4faac service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:07:54.915275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 731/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:08:04.409073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=82 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:08:04.913147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=50 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:09:46.446736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.447397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.447397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.537207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:09:46.538763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:46.654799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 763/1494] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:09:46.666605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.667207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.667207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.692396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:09:46.693601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:46.703177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 732/1495] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:09:46.708142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:46.718700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10372483-4b5c-4696-a136-8f2cd7c8a021 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:46.724986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:46.725716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.726017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.726300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.727049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:09:46.727343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:46.728420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:46.728420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:09:46.728420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:09:46.729360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:46.743200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 764/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:09:46.751791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:46.762904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381380c3-bb93-4929-9785-0caf49b410e3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:46.768427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:46.769062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.769343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.769617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.770301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:46.772125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:46.772125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:09:46.772125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:09:46.773335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:46.816641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 733/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:09:46 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:09:46.821395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:46.832383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551215-f792-4404-9581-3bd913a2cbb1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:46.886230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:09:46.894209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:46.894860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.895166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.895443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.896134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:46.897162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:46.897162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:09:46.897162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:09:46.898366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:46.922762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 765/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:09:46 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:09:46.924002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:46.934350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78359585-a42f-4cc0-ae42-2ff8948fea3a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:46.939555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:46.940179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.940462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.940734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.941469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:46.942589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:46.942589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:09:46.942589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:09:46.943469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:46.958203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 734/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:09:46.963084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:46.974274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dba07-f0a7-4b40-bf0b-6171442772c0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:46.980155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:46.980789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:46.981071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:46.981348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:46.982065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:46.983132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:46.983132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:09:46.983132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:09:46.984008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a7a45-1cc0-4352-a149-9d647be424f6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:46.997993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 766/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:09:47.002527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:47.013656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae8231-ffb2-48f2-9156-cbc0b214d32d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:47.019433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:47.020106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.020392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.020677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] PATH_INFO: `/v3/projects/8f42691d92ea42cbb5f335339e449ac5/users/e399d95f1d0349c09e3f5be6acff74fb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.036022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f42691d92ea42cbb5f335339e449ac5, user_id=e399d95f1d0349c09e3f5be6acff74fb, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:47.037076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:47.037076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:09:47.037076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:09:47.039057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450c59-482b-4859-aac9-b9ea509ada1a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:47.056840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 735/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:09:47 2022] PUT /identity/v3/projects/8f42691d92ea42cbb5f335339e449ac5/users/e399d95f1d0349c09e3f5be6acff74fb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:09:47.069644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1f1d-f429-4a2d-98a3-0ab36166595f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.069970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1f1d-f429-4a2d-98a3-0ab36166595f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.070309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec1f1d-f429-4a2d-98a3-0ab36166595f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.100776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-98ec1f1d-f429-4a2d-98a3-0ab36166595f None None] MFA Rules not processed for user `e399d95f1d0349c09e3f5be6acff74fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:09:47.102008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-98ec1f1d-f429-4a2d-98a3-0ab36166595f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:47.172022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 767/1502] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:09:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:09:47.185801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29999-98b4-4778-b49c-50624315e9cf None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.186092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29999-98b4-4778-b49c-50624315e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.186368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29999-98b4-4778-b49c-50624315e9cf None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.187795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 736/1503] 10.222.0.17 () {54 vars in 964 bytes} [Thu Jul 28 01:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:09:47.193273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d41f4-b5ac-4629-86a3-047cb748348d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.193567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d41f4-b5ac-4629-86a3-047cb748348d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.193841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d41f4-b5ac-4629-86a3-047cb748348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.262403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d74d41f4-b5ac-4629-86a3-047cb748348d None None] MFA Rules not processed for user `6146fcd207164df18b9c08e8fcc1adb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:09:47.263585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d74d41f4-b5ac-4629-86a3-047cb748348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:47.344011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 768/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Jul 28 01:09:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:09:47.352029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac29999-98b4-4778-b49c-50624315e9cf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:47.363200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac29999-98b4-4778-b49c-50624315e9cf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:47.367062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:47.367674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.367952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.368227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.376937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:47.378357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:47.387482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 737/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:09:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:09:47.436222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ba2f-dd95-4c7a-8c69-27ac45e2df38 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.436518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ba2f-dd95-4c7a-8c69-27ac45e2df38 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.436796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153ba2f-dd95-4c7a-8c69-27ac45e2df38 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.438529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 769/1506] 10.222.0.17 () {52 vars in 830 bytes} [Thu Jul 28 01:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:09:47.442509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:47.516697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302142f3-dea9-4e71-8eed-6bddcd264432 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:47.522006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:47.522719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.523041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.523337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] PATH_INFO: `/v3/users/e399d95f1d0349c09e3f5be6acff74fb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.524598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e399d95f1d0349c09e3f5be6acff74fb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:47.525747 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:47.525747 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:09:47.525747 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:09:47.527002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:47.605736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e399d95f1d0349c09e3f5be6acff74fb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:47.619844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e399d95f1d0349c09e3f5be6acff74fb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:47.620600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e399d95f1d0349c09e3f5be6acff74fb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:47.642696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e399d95f1d0349c09e3f5be6acff74fb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:47.659145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:09:47.667109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e399d95f1d0349c09e3f5be6acff74fb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:47.678093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 738/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:09:47 2022] DELETE /identity/v3/users/e399d95f1d0349c09e3f5be6acff74fb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:09:47.683431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.683976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.683976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.703917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:09:47.705521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:47.715264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:09:47.772147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 770/1508] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:09:47.782631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:47.784403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:47.874204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa7c71-7308-4035-8108-f3501392ec66 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:47.878863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3d62-71cc-4f2a-bae1-0cbb0433ff16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:47.879539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3d62-71cc-4f2a-bae1-0cbb0433ff16 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:47.880194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3d62-71cc-4f2a-bae1-0cbb0433ff16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:47.880449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3d62-71cc-4f2a-bae1-0cbb0433ff16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:47.891923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a3d62-71cc-4f2a-bae1-0cbb0433ff16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:47.893424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a3d62-71cc-4f2a-bae1-0cbb0433ff16 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:47.906055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 739/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:09:48.326357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:09:48.328012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:09:48.340330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389ebcd-177e-4ac4-b89c-8bfe5b71bcaf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:09:48.345617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:09:48.346295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:09:48.346582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:09:48.347480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] PATH_INFO: `/v3/projects/8f42691d92ea42cbb5f335339e449ac5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:09:48.353251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f42691d92ea42cbb5f335339e449ac5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:09:48.354320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:09:48.354320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:09:48.354320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:09:48.355646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:09:48.427021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f42691d92ea42cbb5f335339e449ac5', 'request_id': 'req-f32766e2-a230-4af6-8ded-7dd9ac5bc663'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:48.440706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f42691d92ea42cbb5f335339e449ac5', 'request_id': 'req-f32766e2-a230-4af6-8ded-7dd9ac5bc663'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:48.453892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f42691d92ea42cbb5f335339e449ac5', 'request_id': 'req-f32766e2-a230-4af6-8ded-7dd9ac5bc663'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:48.467388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] The token cache is being invalidate because project 8f42691d92ea42cbb5f335339e449ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:09:48.467682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:48.468432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f32766e2-a230-4af6-8ded-7dd9ac5bc663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:09:48.470187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 771/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:09:48 2022] DELETE /identity/v3/projects/8f42691d92ea42cbb5f335339e449ac5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:09:57.899629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=38 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:09:58.465826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:11:26.683557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5535aab-b6fb-4e92-8954-28f25ee45409 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:26.684238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5535aab-b6fb-4e92-8954-28f25ee45409 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:26.684338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5535aab-b6fb-4e92-8954-28f25ee45409 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:26.686127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 740/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:26.693178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaba4b-936e-42c5-89fb-389dc1dcbbb6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:26.693779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaba4b-936e-42c5-89fb-389dc1dcbbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:26.693779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaba4b-936e-42c5-89fb-389dc1dcbbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:26.715689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-79aaba4b-936e-42c5-89fb-389dc1dcbbb6 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:26.716967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-79aaba4b-936e-42c5-89fb-389dc1dcbbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:26.783560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 772/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:26.936724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5535aab-b6fb-4e92-8954-28f25ee45409 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:26.938634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a5535aab-b6fb-4e92-8954-28f25ee45409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:27.063254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5535aab-b6fb-4e92-8954-28f25ee45409 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:27.069320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d109db-f4d2-4e2b-8366-115ac35df286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:27.069987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d109db-f4d2-4e2b-8366-115ac35df286 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:27.070275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d109db-f4d2-4e2b-8366-115ac35df286 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:27.070535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d109db-f4d2-4e2b-8366-115ac35df286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:27.080307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d109db-f4d2-4e2b-8366-115ac35df286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:27.082058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d109db-f4d2-4e2b-8366-115ac35df286 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:27.092910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 741/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:27.954168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1a5a9-dfe7-4439-92cd-7e5c0fe42e05 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:27.954802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1a5a9-dfe7-4439-92cd-7e5c0fe42e05 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:27.954802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1a5a9-dfe7-4439-92cd-7e5c0fe42e05 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:27.956200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 773/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:27.961207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e05ec-863e-40b2-b5b3-feb69ace3e15 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:27.961717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e05ec-863e-40b2-b5b3-feb69ace3e15 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:27.961717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e05ec-863e-40b2-b5b3-feb69ace3e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:27.982240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-913e05ec-863e-40b2-b5b3-feb69ace3e15 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:27.984175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-913e05ec-863e-40b2-b5b3-feb69ace3e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:27.993132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 742/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:28.150480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1a5a9-dfe7-4439-92cd-7e5c0fe42e05 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:28.170879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1a5a9-dfe7-4439-92cd-7e5c0fe42e05 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:28.178409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818eb40-e951-4f32-88ce-1254856829e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:28.179307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818eb40-e951-4f32-88ce-1254856829e0 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:28.179732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818eb40-e951-4f32-88ce-1254856829e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:28.180045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818eb40-e951-4f32-88ce-1254856829e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:28.193422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818eb40-e951-4f32-88ce-1254856829e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:28.195226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818eb40-e951-4f32-88ce-1254856829e0 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:28.206641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 774/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:28.648366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad5d0-8ba6-42bb-8ef1-bee87be48bc6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:28.648652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad5d0-8ba6-42bb-8ef1-bee87be48bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:28.648935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad5d0-8ba6-42bb-8ef1-bee87be48bc6 None None] PATH_INFO: `/v3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:28.650274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 743/1517] 10.222.0.42 () {54 vars in 925 bytes} [Thu Jul 28 01:11:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:11:28.657509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc375f-1d1c-47b5-8fe9-e354f308d9a1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:28.657805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc375f-1d1c-47b5-8fe9-e354f308d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:28.658142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc375f-1d1c-47b5-8fe9-e354f308d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:28.716482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c3bc375f-1d1c-47b5-8fe9-e354f308d9a1 None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:28.717722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c3bc375f-1d1c-47b5-8fe9-e354f308d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:28.790934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 775/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jul 28 01:11:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:32.646753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee7bfa-2a94-465f-884b-71313d9810ca None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:32.647357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee7bfa-2a94-465f-884b-71313d9810ca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:32.647357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee7bfa-2a94-465f-884b-71313d9810ca None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:32.648587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 744/1519] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jul 28 01:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:32.654254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748750-f2f7-4041-9a10-8323ba1ac6f3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:32.655063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748750-f2f7-4041-9a10-8323ba1ac6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:32.655063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748750-f2f7-4041-9a10-8323ba1ac6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:32.677549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-55748750-f2f7-4041-9a10-8323ba1ac6f3 None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:32.678829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-55748750-f2f7-4041-9a10-8323ba1ac6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:32.689178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 776/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jul 28 01:11:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:32.697536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee7bfa-2a94-465f-884b-71313d9810ca None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:32.712800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee7bfa-2a94-465f-884b-71313d9810ca None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:32.718135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244eef64-1231-4429-bd7d-9cd7763bc68a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:32.718779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eef64-1231-4429-bd7d-9cd7763bc68a service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:32.719099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eef64-1231-4429-bd7d-9cd7763bc68a service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:32.719387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eef64-1231-4429-bd7d-9cd7763bc68a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:32.729169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244eef64-1231-4429-bd7d-9cd7763bc68a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:32.730804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244eef64-1231-4429-bd7d-9cd7763bc68a service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:32.741648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 745/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:39.500965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a68361-a80c-4014-a4d8-0baf744bc2f2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:39.501542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a68361-a80c-4014-a4d8-0baf744bc2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:39.501542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a68361-a80c-4014-a4d8-0baf744bc2f2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:39.503594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 777/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:39.510614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea264a72-92f6-4650-8032-7205bafdd611 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:39.511153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea264a72-92f6-4650-8032-7205bafdd611 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:39.511153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea264a72-92f6-4650-8032-7205bafdd611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:39.532033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ea264a72-92f6-4650-8032-7205bafdd611 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:39.533492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ea264a72-92f6-4650-8032-7205bafdd611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:39.541857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 746/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:39.696108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a68361-a80c-4014-a4d8-0baf744bc2f2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:39.711882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a68361-a80c-4014-a4d8-0baf744bc2f2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:39.719012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe8ab4-db27-4df5-9512-4ae0f4551b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:39.719632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe8ab4-db27-4df5-9512-4ae0f4551b82 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:39.719925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe8ab4-db27-4df5-9512-4ae0f4551b82 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:39.720196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe8ab4-db27-4df5-9512-4ae0f4551b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:39.729295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe8ab4-db27-4df5-9512-4ae0f4551b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:39.730857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe8ab4-db27-4df5-9512-4ae0f4551b82 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:39.741367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 778/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:40.590137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5695256-b620-4082-91bb-c41503670b9c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:40.590734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5695256-b620-4082-91bb-c41503670b9c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:40.590734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5695256-b620-4082-91bb-c41503670b9c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:40.592067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 747/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 01:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:40.597583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747acb9-d3b5-453a-aeb0-b55f63e8969f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:40.598177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747acb9-d3b5-453a-aeb0-b55f63e8969f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:40.598177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747acb9-d3b5-453a-aeb0-b55f63e8969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:40.627988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d747acb9-d3b5-453a-aeb0-b55f63e8969f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:40.630288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d747acb9-d3b5-453a-aeb0-b55f63e8969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:40.644068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 779/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 01:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:40.806280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5695256-b620-4082-91bb-c41503670b9c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:40.821931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5695256-b620-4082-91bb-c41503670b9c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:40.827926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e15fd-8cb0-4e37-a8c9-81dc8bef7f7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:40.828584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e15fd-8cb0-4e37-a8c9-81dc8bef7f7e service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:40.828867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e15fd-8cb0-4e37-a8c9-81dc8bef7f7e service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:40.829148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e15fd-8cb0-4e37-a8c9-81dc8bef7f7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:40.838889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e15fd-8cb0-4e37-a8c9-81dc8bef7f7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:40.840584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e15fd-8cb0-4e37-a8c9-81dc8bef7f7e service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:40.851762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 748/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:45.216795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb14f8-c88d-41bd-a36a-9c3513c6a49a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:45.217396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb14f8-c88d-41bd-a36a-9c3513c6a49a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:45.217396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb14f8-c88d-41bd-a36a-9c3513c6a49a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:45.218671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 780/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:45.225994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133d1b-b558-44e2-8173-443077d55a62 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:45.226704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133d1b-b558-44e2-8173-443077d55a62 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:45.227074 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133d1b-b558-44e2-8173-443077d55a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:45.258403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fd133d1b-b558-44e2-8173-443077d55a62 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:45.259664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd133d1b-b558-44e2-8173-443077d55a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:45.267816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 749/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:45.357165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb14f8-c88d-41bd-a36a-9c3513c6a49a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:45.359231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3acb14f8-c88d-41bd-a36a-9c3513c6a49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:45.394239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:11:45.430429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb14f8-c88d-41bd-a36a-9c3513c6a49a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:45.435433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb9b1a-49b2-4461-b84c-e573b9d7da7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:45.436088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9b1a-49b2-4461-b84c-e573b9d7da7d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:45.436368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9b1a-49b2-4461-b84c-e573b9d7da7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:45.436640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9b1a-49b2-4461-b84c-e573b9d7da7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:45.448315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb9b1a-49b2-4461-b84c-e573b9d7da7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:45.449775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb9b1a-49b2-4461-b84c-e573b9d7da7d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:45.462658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 781/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:47.606766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1137d3f-8a61-48fe-9ba9-fa582ec6bf87 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:47.607526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1137d3f-8a61-48fe-9ba9-fa582ec6bf87 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:47.607526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1137d3f-8a61-48fe-9ba9-fa582ec6bf87 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:47.608807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 750/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:11:47.615286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615f50b-e3d3-4918-8657-c3e0d8702e6f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:47.616174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615f50b-e3d3-4918-8657-c3e0d8702e6f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:47.616174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615f50b-e3d3-4918-8657-c3e0d8702e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:47.640045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7615f50b-e3d3-4918-8657-c3e0d8702e6f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:47.641341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7615f50b-e3d3-4918-8657-c3e0d8702e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:47.651575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 782/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:47.689746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1137d3f-8a61-48fe-9ba9-fa582ec6bf87 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:47.702658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1137d3f-8a61-48fe-9ba9-fa582ec6bf87 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:47.707846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c490d3-7ac4-418d-993d-29ba2795d7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:47.708714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c490d3-7ac4-418d-993d-29ba2795d7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:47.709140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c490d3-7ac4-418d-993d-29ba2795d7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:47.709552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c490d3-7ac4-418d-993d-29ba2795d7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:47.720209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c490d3-7ac4-418d-993d-29ba2795d7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:47.721749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c490d3-7ac4-418d-993d-29ba2795d7d2 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:47.733286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 751/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:51.306326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7002d15-cd04-47b5-9257-bd005702209e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:51.306918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7002d15-cd04-47b5-9257-bd005702209e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:51.307118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7002d15-cd04-47b5-9257-bd005702209e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:51.309381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 783/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:11:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:51.315272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856beb4-76ae-44eb-b4f7-3322ef304424 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:51.315917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856beb4-76ae-44eb-b4f7-3322ef304424 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:51.315917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856beb4-76ae-44eb-b4f7-3322ef304424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:51.324054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=74 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:11:51.342372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f856beb4-76ae-44eb-b4f7-3322ef304424 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:51.344080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f856beb4-76ae-44eb-b4f7-3322ef304424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:51.352453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 752/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:51.663037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7002d15-cd04-47b5-9257-bd005702209e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:51.664957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b7002d15-cd04-47b5-9257-bd005702209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:51.734291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7002d15-cd04-47b5-9257-bd005702209e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:51.738878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dfdbf-42b9-49af-b24e-03b3bb44e6fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:51.739524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dfdbf-42b9-49af-b24e-03b3bb44e6fe service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:51.739814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dfdbf-42b9-49af-b24e-03b3bb44e6fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:51.740097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dfdbf-42b9-49af-b24e-03b3bb44e6fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:51.749925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dfdbf-42b9-49af-b24e-03b3bb44e6fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:51.751416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dfdbf-42b9-49af-b24e-03b3bb44e6fe service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:51.762325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 784/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:11:53.804512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9f69-f522-4f17-b9c9-1fdfa3640861 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:53.805071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9f69-f522-4f17-b9c9-1fdfa3640861 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:53.805071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9f69-f522-4f17-b9c9-1fdfa3640861 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:53.806372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 753/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 01:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:11:53.811983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ddbe-5295-456b-b70a-d405541204d9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:53.812447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ddbe-5295-456b-b70a-d405541204d9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:53.812555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ddbe-5295-456b-b70a-d405541204d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:53.836804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-baa6ddbe-5295-456b-b70a-d405541204d9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:11:53.838045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-baa6ddbe-5295-456b-b70a-d405541204d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:53.847533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 785/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 01:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:11:54.162541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9f69-f522-4f17-b9c9-1fdfa3640861 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:11:54.164621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-be3b9f69-f522-4f17-b9c9-1fdfa3640861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:11:54.179774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9f69-f522-4f17-b9c9-1fdfa3640861 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:11:54.183805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dd73-7241-4473-a1f9-a51350d07ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:11:54.184509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dd73-7241-4473-a1f9-a51350d07ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:11:54.184716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dd73-7241-4473-a1f9-a51350d07ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:11:54.185000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dd73-7241-4473-a1f9-a51350d07ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:11:54.195052 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dd73-7241-4473-a1f9-a51350d07ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:11:54.196525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dd73-7241-4473-a1f9-a51350d07ba2 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:11:54.206424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 754/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:12:03.831724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=47 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:12:04.202031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=8 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:23.395474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:23.396105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:23.396105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:23.439165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:23.441928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:23.463719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 786/1540] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:23.473147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afdfc2-2fec-4081-8115-635be99bb357 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:23.474168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afdfc2-2fec-4081-8115-635be99bb357 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:23.474168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afdfc2-2fec-4081-8115-635be99bb357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:23.528393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e1afdfc2-2fec-4081-8115-635be99bb357 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:23.529752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e1afdfc2-2fec-4081-8115-635be99bb357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:23.546033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 755/1541] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:23.551821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:23.586466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b5397-53e8-4a89-8947-a50ef82e5cb9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:23.599165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:23.600625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:23.601553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:23.602263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:23.604041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:23.606390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a9b9-284a-4be9-a24d-a4a2006ce02b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:23.606671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:23.606867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a9b9-284a-4be9-a24d-a4a2006ce02b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:23.607346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a9b9-284a-4be9-a24d-a4a2006ce02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:23.610330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:23.647370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 787/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:23.655217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:23.668513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2712a9b9-284a-4be9-a24d-a4a2006ce02b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:23.670595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2712a9b9-284a-4be9-a24d-a4a2006ce02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:23.682757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e6017-ab7c-4d0d-bb67-3fb6e7a8b03b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:23.697077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:23.698558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:23.699307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:23.700857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:23.702401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:23.706869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 756/1543] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:23.709305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:13:23.709305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:13:23.709305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:13:23.713227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:23.715720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:23.716402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:23.716879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:23.788750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:23.791427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:23.807729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 788/1544] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:23 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:13:23.815595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:23.826038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 757/1545] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:23.832734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:23.859064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b150b90-428c-4445-ad5f-3ba009dd429f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.038738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9aef7-b365-4d4f-a874-6d6ba9a82ee3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.041884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.043048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.043593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.044079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.045932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:24.046468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.048424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:13:24.048424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:13:24.048424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:13:24.050050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3940c3-3879-4365-851b-f4487bb78bd8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.050427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.051840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.052607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.057345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.059819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.064030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.096201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 758/1546] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:24.099183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.100483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.103181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.132231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 789/1547] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:23 2022] POST /identity/v3/users => generated 290 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:13:24.132231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.153650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d472c-373e-454f-985b-721227a81187 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.169040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:24.172743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:24.180464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a9529-1cb9-48a9-ae2f-6e98b31326e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.181309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a9529-1cb9-48a9-ae2f-6e98b31326e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.181586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a9529-1cb9-48a9-ae2f-6e98b31326e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.181868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a9529-1cb9-48a9-ae2f-6e98b31326e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.182565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a9529-1cb9-48a9-ae2f-6e98b31326e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.184696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a9529-1cb9-48a9-ae2f-6e98b31326e5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.200541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 759/1548] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:24.203953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.234447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b252b3-e9a9-4d90-ac8b-6d67fbf86d02 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.249817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.252358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.253204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.253934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.255892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.259417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.281566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 790/1549] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:24.281905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796d799-9214-4280-8def-c20c056da76b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.282189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796d799-9214-4280-8def-c20c056da76b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.282471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796d799-9214-4280-8def-c20c056da76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.307294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 760/1550] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:24.313210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.343966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c796d799-9214-4280-8def-c20c056da76b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:24.345381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c796d799-9214-4280-8def-c20c056da76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:24.348008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8559c-297c-46a6-ba76-45bc17e3b7ff admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.488383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 791/1551] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:24.489457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796d799-9214-4280-8def-c20c056da76b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.508888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796d799-9214-4280-8def-c20c056da76b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.540543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.541215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.541500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.541780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.542508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.544211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.559001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.561115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.561539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 792/1552] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:24.562073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.562430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.563181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.564921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.573710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:13:24.573710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:13:24.573710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:13:24.577197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.585764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10fd4a-0f09-4813-81ae-d7dbf99da06a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.612539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.613211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.613506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.613783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.615117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:24.615117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.616553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771b7bc-44c5-42b6-8256-82e1c920921f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.643709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 793/1553] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:24.644800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3d4a8-332f-4086-a874-02ad55c4febe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.645093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3d4a8-332f-4086-a874-02ad55c4febe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.645376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3d4a8-332f-4086-a874-02ad55c4febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.659347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 761/1554] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/users => generated 284 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:24.659734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:24.660312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.684195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb5ad-fcf9-44d3-a592-e400c54aae18 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.686288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9bb3d4a8-332f-4086-a874-02ad55c4febe None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:24.687589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9bb3d4a8-332f-4086-a874-02ad55c4febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:24.696887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.698079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.698585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.699144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] PATH_INFO: `/v3/projects/c8b3087180c642a78010ed1382cf1868/users/84b16be4f85b440a9e3afea73ffc7871/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.699401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 794/1555] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:24.700887 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.701299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.705268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.737136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b3087180c642a78010ed1382cf1868, user_id=84b16be4f85b440a9e3afea73ffc7871, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.739058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:24.740513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:24.742544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.751773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 795/1556] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:24.753175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.766880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca3cd4-6b70-4b7b-b029-e76afeb6ce09 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.771119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 762/1557] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:24 2022] PUT /identity/v3/projects/c8b3087180c642a78010ed1382cf1868/users/84b16be4f85b440a9e3afea73ffc7871/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:24.772873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:24.825762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:24.861089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c49b4-a297-4508-91e9-ed3d261163ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:24.862266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c49b4-a297-4508-91e9-ed3d261163ed admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.862722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c49b4-a297-4508-91e9-ed3d261163ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.863226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c49b4-a297-4508-91e9-ed3d261163ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.864121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c49b4-a297-4508-91e9-ed3d261163ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:24.866421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c49b4-a297-4508-91e9-ed3d261163ed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:24.903291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 796/1558] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:24.909368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eda0d4-73f4-4cdd-b37f-2a4566d3f74b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:24.909961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eda0d4-73f4-4cdd-b37f-2a4566d3f74b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:24.910402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eda0d4-73f4-4cdd-b37f-2a4566d3f74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:24.916105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564a13-15ae-4982-a9a2-95bc1015d221 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:24.936572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-22eda0d4-73f4-4cdd-b37f-2a4566d3f74b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:24.938343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-22eda0d4-73f4-4cdd-b37f-2a4566d3f74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.017983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 797/1559] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.027266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c9fba-1cab-4375-a00c-ed6c4e682b4a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.027555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c9fba-1cab-4375-a00c-ed6c4e682b4a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.027945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c9fba-1cab-4375-a00c-ed6c4e682b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.042136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876303da-1288-443d-8340-cb759f256aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.043451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876303da-1288-443d-8340-cb759f256aac admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.044089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876303da-1288-443d-8340-cb759f256aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.044713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876303da-1288-443d-8340-cb759f256aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.046211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876303da-1288-443d-8340-cb759f256aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.049547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876303da-1288-443d-8340-cb759f256aac admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.049961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-251c9fba-1cab-4375-a00c-ed6c4e682b4a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.051288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-251c9fba-1cab-4375-a00c-ed6c4e682b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.061218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 798/1560] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.065627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.066132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.066427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.075845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 763/1561] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:25.075845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d00b-6d6b-42dc-8933-156551f2d3bf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.076153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d00b-6d6b-42dc-8933-156551f2d3bf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.077020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d00b-6d6b-42dc-8933-156551f2d3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.093679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.095047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.103664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 799/1562] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.108696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.111370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-abf6d00b-6d6b-42dc-8933-156551f2d3bf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.113067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-abf6d00b-6d6b-42dc-8933-156551f2d3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.119772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb20ec-5abe-44e6-9fc4-84b9c7df57f9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.124501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed34756-e2cb-4ee6-bf01-e53610ef5c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.125344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed34756-e2cb-4ee6-bf01-e53610ef5c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.125597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed34756-e2cb-4ee6-bf01-e53610ef5c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.125900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed34756-e2cb-4ee6-bf01-e53610ef5c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.129599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 764/1563] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.130166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed34756-e2cb-4ee6-bf01-e53610ef5c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.130778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.130898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed34756-e2cb-4ee6-bf01-e53610ef5c39 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.130986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.130986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.163484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 800/1564] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/users => generated 285 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:25.163744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.165443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.166039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.166410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.167004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.180941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 765/1565] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.182553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.189628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.191281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.200562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 801/1566] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.205282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.205491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ece29-16ee-463f-9024-0489a9cb3161 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.210368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.211892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.212176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.212459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.213490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:25.213799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.215468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.215818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc9a72-59df-46a3-af1a-36ff8a662220 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.221930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.222730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.223177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.223563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.224446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.226430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.231453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 766/1567] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:25.232478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.256288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 802/1568] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:25.256288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.257236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c8640-323b-4438-bf9a-2cda2052b645 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.268838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.269824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.270130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.270429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.271462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:25.271771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.273525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.286811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 767/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:25.287841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.298780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969511e-dd24-4a5d-a56a-c1b53f7119db admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.303920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f50d02-0bb9-463b-a07d-330e81080338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.304670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f50d02-0bb9-463b-a07d-330e81080338 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.305566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f50d02-0bb9-463b-a07d-330e81080338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.305855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f50d02-0bb9-463b-a07d-330e81080338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.306575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f50d02-0bb9-463b-a07d-330e81080338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.307907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:25.308235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f50d02-0bb9-463b-a07d-330e81080338 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.320775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 768/1570] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:25.322116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.322442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.322727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.345842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8659764-0f60-408a-8319-762f6a5f1d65 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.346108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.347326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.350321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.351075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.351362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.351647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.356518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 769/1571] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.357387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.369102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.370082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58caee3-2552-4c19-a566-8fde16cebfac None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.370582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.375227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.375908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.376881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.376881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.377292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:25.377533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.379641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.382294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 803/1572] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:25.383418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.395002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b131597-8333-47d6-93d3-3833c9a8317a service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.396232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 770/1573] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:25.397604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.400527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c7a1-0999-46e8-86a0-569c6c217559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.401177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c7a1-0999-46e8-86a0-569c6c217559 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.401462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c7a1-0999-46e8-86a0-569c6c217559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.401755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c7a1-0999-46e8-86a0-569c6c217559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.402459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20c7a1-0999-46e8-86a0-569c6c217559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.404697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20c7a1-0999-46e8-86a0-569c6c217559 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.410449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bae06-300b-4658-905d-66dab7cf0012 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.415448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.416257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.416603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.416942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] PATH_INFO: `/v3/projects/ba821829826b4d499eb669af047a14d9/users/4c9c7d609b4a473193d7ed97c4b7ffd1/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.447772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 804/1574] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:25.447772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.447772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.447772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.453407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba821829826b4d499eb669af047a14d9, user_id=4c9c7d609b4a473193d7ed97c4b7ffd1, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.461656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.469965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.474011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.490878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 771/1575] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:25 2022] PUT /identity/v3/projects/ba821829826b4d499eb669af047a14d9/users/4c9c7d609b4a473193d7ed97c4b7ffd1/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:25.490878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.538231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:25.551871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 805/1576] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.559345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.569661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14583b-201e-4345-8c16-fab0a4f1f5a8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.581856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70804b7-da2e-40ca-adca-ad774ad1f20a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.589635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.590525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.590886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.591698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.592477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.596476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.639018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 772/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:25.639936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.657243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.658019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.658429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.658766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.659585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.661272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.673641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 806/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:25.674618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.687850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48740788-677a-43b0-a3b4-5bf9c1cee019 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.711984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50edd5f-64aa-432c-9f09-062a6a0a3d4d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.716981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.717627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.717912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.718193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.718931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:25.720280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.720967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.734585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 773/1579] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:25.735497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.747902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:25.751143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9ac3f-1a82-499e-a3a2-0dc4eab0cfba admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.759207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38952e5-3a10-4176-beaf-a06a50d2cf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.760014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38952e5-3a10-4176-beaf-a06a50d2cf86 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.760297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38952e5-3a10-4176-beaf-a06a50d2cf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.760575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38952e5-3a10-4176-beaf-a06a50d2cf86 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.761259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38952e5-3a10-4176-beaf-a06a50d2cf86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.763429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38952e5-3a10-4176-beaf-a06a50d2cf86 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.769675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad6800-a1b6-4f9d-ad2f-a96696f4e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.770387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6800-a1b6-4f9d-ad2f-a96696f4e369 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.770736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6800-a1b6-4f9d-ad2f-a96696f4e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.771105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6800-a1b6-4f9d-ad2f-a96696f4e369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.771874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad6800-a1b6-4f9d-ad2f-a96696f4e369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.774154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad6800-a1b6-4f9d-ad2f-a96696f4e369 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.797820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 774/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/users => generated 295 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:25.799207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.799495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.799784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.828394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 807/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:25.831304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.831753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.832202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.834196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] MFA Rules not processed for user `4c9c7d609b4a473193d7ed97c4b7ffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.835531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.855964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:25.857576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:25.915702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 775/1582] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.915702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.941811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 808/1583] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:25.941811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:25.949271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab11f6e-3c78-4840-8cef-0e31ad4ca104 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:25.951946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:25.955716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:25.956421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:25.956709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:25.956981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] PATH_INFO: `/v3/projects/3f9548d25b094b9a9d5418410c044445/users/59c4bd188120450fa69be750c9308521/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:25.973563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9548d25b094b9a9d5418410c044445, user_id=59c4bd188120450fa69be750c9308521, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:25.974612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:13:25.974612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:13:25.974612 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:13:25.977097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:25.992891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 809/1584] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:25 2022] PUT /identity/v3/projects/3f9548d25b094b9a9d5418410c044445/users/59c4bd188120450fa69be750c9308521/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:25.994398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.007668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02291d-7d7e-4dac-b08a-151ccc774bd0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.076374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eeab2-4e2e-453d-a06f-7143b9e606a6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.081446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.082104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.082439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.082716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.083460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.085688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.092698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.093461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.093816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.094685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.094956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.097487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.121509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 810/1585] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:26.122525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.124564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 776/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:25 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:26.125748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.137019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48d632-fe24-45c0-b5cc-986d4517d613 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.177491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:26.231412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f66a-c8c1-4ef6-85c6-e03046050fc2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.240685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.241997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.242343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.242622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.243520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.245528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.271538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 811/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:26.271538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.279597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.280891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.281358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.283106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.284130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.289062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.346718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 777/1588] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:26 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:26.348298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.362988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07ecd-9042-447b-adaf-0cd3ab56577e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.372711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.384697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.385314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.385745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.387441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:26.387993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.390790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e609fae-b53c-4f2b-8f97-e5377350cdef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.406281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c060f9c-ac07-4e04-8231-505077f1d744 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.411521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.412902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.413297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.413909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.427584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 778/1589] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:26.429404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.432435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.433045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.515245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.516496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] MFA Rules not processed for user `59c4bd188120450fa69be750c9308521`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:26.517004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.518171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:26.532929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 812/1590] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:26.533906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.549214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5995e3-129f-4a58-84cd-5d7ad3b3d227 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.557031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.558061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.558500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.558860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.560442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.563231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.589626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:26.617852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 813/1591] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:26 2022] POST /identity/v3/users => generated 286 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:26.619013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.683261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bff7e-5dcf-409f-8b56-0c971707b8e5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.706333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.707034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.707324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.713693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.714470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.717011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.738306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 779/1592] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:26.745658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.760437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 814/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:26.761970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.773850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037f4a-ffa4-45ab-84a9-e09fe156e918 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.785783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.786476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.786763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.787736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae6d90-96f3-4a2f-8a1b-78d8547d1a51 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.788789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.789549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.794031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.801066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.802401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.803499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.805778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.806562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.812364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.836295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 780/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:26.837272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.853747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 815/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:26.853747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.863172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07010aab-d563-432f-bacb-4358066fc8a3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.874916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.876205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.876205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.876465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.877073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.879035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04810151-2239-4f98-800c-0aa1132c5c1a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.880810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.890611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:26.891367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:26.891656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:26.892000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:26.893080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:26.896388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:26.918813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 816/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:26.922747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:26.944747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18481c-629a-46ad-ba6e-41649177711f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:26.946967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 781/1597] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:26 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:26.947891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.029522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:27.070145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eee52-e35e-4e0c-92c2-1d2a20210651 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.075753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.076585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.077491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.077491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.078012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.079704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.082164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.082857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.083149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.083475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.084170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.086238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.108113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 782/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:27.113161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.124583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 817/1599] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:26 2022] POST /identity/v3/projects => generated 369 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:27.126727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.142470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac5a9d-4484-491c-9ff4-33533208962f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.230343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.232517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.233295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.234152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] PATH_INFO: `/v3/projects/6e3df8020ac0427bae99a7fc30bdfa48/users/166cef282cb24a5aaef0ca8b28f02e56/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.253817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc31ed-055b-46e1-88a9-fe21a05217f2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.265993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.266308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.267157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.267467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.268197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.270187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.284532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 818/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:27.285758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.298102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812d263-5281-470a-bdf1-e2539149c213 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.303802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.304509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.304784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.305052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.306026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.307261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3df8020ac0427bae99a7fc30bdfa48, user_id=166cef282cb24a5aaef0ca8b28f02e56, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.307851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.325737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.336707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 819/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:27.345672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.370955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3d03-24ad-44c2-8c91-1487c246d6ec admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.372766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 783/1602] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:27 2022] PUT /identity/v3/projects/6e3df8020ac0427bae99a7fc30bdfa48/users/166cef282cb24a5aaef0ca8b28f02e56/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:27.374875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.405044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:27.561374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b882ad4-039b-4625-99d1-152cbda46ba2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.570171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.571617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.572540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.573929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.575499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.587127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.589710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.590481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.590772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.591116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] PATH_INFO: `/v3/projects/af4689d2f0ef4d71a1c465610011f2d7/users/018eb80709e94d5e8653fb6eeb342c54/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.630118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4689d2f0ef4d71a1c465610011f2d7, user_id=018eb80709e94d5e8653fb6eeb342c54, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.630118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.638967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 820/1603] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:27 2022] POST /identity/v3/users => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:27.638967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.656556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd804c68-d058-4d6d-b92a-e78ab5af3d21 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.665997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 784/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:27 2022] PUT /identity/v3/projects/af4689d2f0ef4d71a1c465610011f2d7/users/018eb80709e94d5e8653fb6eeb342c54/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:27.667015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.716521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:27.760957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.762471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.763782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.764489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.766102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.770549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.785317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9afa3-a585-4762-8610-bfbd3371cd39 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.794865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.796123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.796826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.797484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] PATH_INFO: `/v3/projects/347bcdc7636e4f93af2597f853926b13/users/c8ab1df6f15143de8f922a626317fa9c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.808696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 821/1605] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:27 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:27.809733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.825581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1759581-4ffb-4b2d-bc84-2fb9d03f4c18 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:27.830387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843d3ed-2113-4ae1-b7b5-e751e25dc953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:27.833264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347bcdc7636e4f93af2597f853926b13, user_id=c8ab1df6f15143de8f922a626317fa9c, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.833412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843d3ed-2113-4ae1-b7b5-e751e25dc953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.833412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843d3ed-2113-4ae1-b7b5-e751e25dc953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.833412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843d3ed-2113-4ae1-b7b5-e751e25dc953 admin admin] PATH_INFO: `/v3/projects/8fe8407ac0834896b8693848d6c023d8/users/3fcfb45bc3bc48ec9c6cdd33d18b66b0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.834074 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.893717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843d3ed-2113-4ae1-b7b5-e751e25dc953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe8407ac0834896b8693848d6c023d8, user_id=3fcfb45bc3bc48ec9c6cdd33d18b66b0, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:27.893717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843d3ed-2113-4ae1-b7b5-e751e25dc953 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:27.893717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 822/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:27 2022] PUT /identity/v3/projects/8fe8407ac0834896b8693848d6c023d8/users/3fcfb45bc3bc48ec9c6cdd33d18b66b0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:27.893717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74898-36d3-48a2-a3a9-a5c7fc88d57c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:27.893717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74898-36d3-48a2-a3a9-a5c7fc88d57c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:27.893717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b74898-36d3-48a2-a3a9-a5c7fc88d57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:27.897448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 785/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:27 2022] PUT /identity/v3/projects/347bcdc7636e4f93af2597f853926b13/users/c8ab1df6f15143de8f922a626317fa9c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:27.897448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:27.920331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d5b74898-36d3-48a2-a3a9-a5c7fc88d57c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:27.930167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d5b74898-36d3-48a2-a3a9-a5c7fc88d57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:28.020562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665064d-1645-492d-938c-37e385dc76a4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.048086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 823/1608] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:28.050097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.050524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.050810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.063257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:28.074643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:28.076874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:28.091704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 824/1609] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:28.091704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.104471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94431831-a09a-4a87-ad1a-f2a863d13ded None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.111857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eccc1-7745-4dcb-9f5d-2fd2acd4801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.113016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eccc1-7745-4dcb-9f5d-2fd2acd4801e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.113491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eccc1-7745-4dcb-9f5d-2fd2acd4801e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.113952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eccc1-7745-4dcb-9f5d-2fd2acd4801e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.115056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eccc1-7745-4dcb-9f5d-2fd2acd4801e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.117321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eccc1-7745-4dcb-9f5d-2fd2acd4801e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.121963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaac819-e114-49ca-8ef1-5c4410fce0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.122786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaac819-e114-49ca-8ef1-5c4410fce0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.123304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaac819-e114-49ca-8ef1-5c4410fce0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.123707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaac819-e114-49ca-8ef1-5c4410fce0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.124772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaac819-e114-49ca-8ef1-5c4410fce0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.126679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaac819-e114-49ca-8ef1-5c4410fce0d6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.134877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 825/1610] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:28.135602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.136012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.136735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.145284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 786/1611] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:28.145284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.145284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.145284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.159161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:28.160415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:28.170354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 826/1612] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:28.171088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.175964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:28.181430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] MFA Rules not processed for user `3fcfb45bc3bc48ec9c6cdd33d18b66b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:28.183103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:28.249195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7d5f-2063-4566-8baf-6786d453abee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.254193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.255420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.255709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.255990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.267091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.268633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.276510 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 787/1613] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:28.282388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.283188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 827/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:28.294306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.294505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340e672-15e4-4bf7-8364-a1a74946c14c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.298674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.299470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.299799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.300084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.315823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.316373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0764d-9df1-4c06-a810-25d33007644d service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.319329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.329616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.330364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.330654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.331185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] PATH_INFO: `/v3/projects/4ab8a4d21b014c59874e3a80f3e5d36e/users/490388172bdd4b9ea44267169746856c/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.345142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 788/1615] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:28.348787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.363534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:28.368799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cbd96-b375-4e8a-9dca-ba0ea0192aab service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.371515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab8a4d21b014c59874e3a80f3e5d36e, user_id=490388172bdd4b9ea44267169746856c, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.374796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.379156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aaf6b-c04e-4352-80f6-4a5d4854a490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.380473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aaf6b-c04e-4352-80f6-4a5d4854a490 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.380764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aaf6b-c04e-4352-80f6-4a5d4854a490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.381764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aaf6b-c04e-4352-80f6-4a5d4854a490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.382495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aaf6b-c04e-4352-80f6-4a5d4854a490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.386014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aaf6b-c04e-4352-80f6-4a5d4854a490 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.401462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 828/1616] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:28 2022] PUT /identity/v3/projects/4ab8a4d21b014c59874e3a80f3e5d36e/users/490388172bdd4b9ea44267169746856c/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:28.404765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.407440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 789/1617] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:28.409751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a596d3-00a7-4014-83d7-9d703e6992e1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.410358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a596d3-00a7-4014-83d7-9d703e6992e1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.411064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a596d3-00a7-4014-83d7-9d703e6992e1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.420778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 790/1618] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 01:13:28 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:28.421767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a596d3-00a7-4014-83d7-9d703e6992e1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.505599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354d49-12e4-46de-88ee-c73523bdbefd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.514203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.515006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.515344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.515984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.612888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a596d3-00a7-4014-83d7-9d703e6992e1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.636273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9dce4c-7709-487b-85e3-325091adc12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.638435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dce4c-7709-487b-85e3-325091adc12d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.640338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dce4c-7709-487b-85e3-325091adc12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.641652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dce4c-7709-487b-85e3-325091adc12d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.642596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9dce4c-7709-487b-85e3-325091adc12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.647449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9dce4c-7709-487b-85e3-325091adc12d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.648560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.650371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.678442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 829/1619] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:28.679467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 791/1620] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:28.680021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.684553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffdcf8-f839-418f-9165-022299f775a3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.684872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffdcf8-f839-418f-9165-022299f775a3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.685114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffdcf8-f839-418f-9165-022299f775a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.701585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8563f-0013-460b-b6c5-26d0c0c9c10e service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.720786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.721512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.721800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.722092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] PATH_INFO: `/v3/projects/74152caac2cf4f8abdd7855f775ec5ce/users/302f4cf1c97941a09dd1734ee549825d/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.761310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74152caac2cf4f8abdd7855f775ec5ce, user_id=302f4cf1c97941a09dd1734ee549825d, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.764297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.784947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fdffdcf8-f839-418f-9165-022299f775a3 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:28.788866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fdffdcf8-f839-418f-9165-022299f775a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:28.796276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 830/1621] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:28 2022] PUT /identity/v3/projects/74152caac2cf4f8abdd7855f775ec5ce/users/302f4cf1c97941a09dd1734ee549825d/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:28.796276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:28.799500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:28.941086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 792/1622] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:28.942941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc040a9-966d-4c3d-8037-bf67c7faba55 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.943340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc040a9-966d-4c3d-8037-bf67c7faba55 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.943724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc040a9-966d-4c3d-8037-bf67c7faba55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.951782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6f0c-513b-461c-a6ac-4a35e18b8a35 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:28.969053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:28.969773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:28.970065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:28.970347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] PATH_INFO: `/v3/projects/4ab8a4d21b014c59874e3a80f3e5d36e/users/490388172bdd4b9ea44267169746856c/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:28.982633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab8a4d21b014c59874e3a80f3e5d36e, user_id=490388172bdd4b9ea44267169746856c, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:28.982633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:28.991253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2bc040a9-966d-4c3d-8037-bf67c7faba55 None None] MFA Rules not processed for user `302f4cf1c97941a09dd1734ee549825d`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:28.993133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2bc040a9-966d-4c3d-8037-bf67c7faba55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:29.037634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 831/1623] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:28 2022] PUT /identity/v3/projects/4ab8a4d21b014c59874e3a80f3e5d36e/users/490388172bdd4b9ea44267169746856c/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:29.038640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:29.083220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:29.131329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 793/1624] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:29.134800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d40d7-6da8-43bc-8d9e-cb617a5416a4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.137119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d40d7-6da8-43bc-8d9e-cb617a5416a4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.137454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d40d7-6da8-43bc-8d9e-cb617a5416a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.228959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9e93-7704-4577-a96d-1aedf4b55ea7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:29.232568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7c3d40d7-6da8-43bc-8d9e-cb617a5416a4 None None] MFA Rules not processed for user `490388172bdd4b9ea44267169746856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:29.238270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c3d40d7-6da8-43bc-8d9e-cb617a5416a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:29.241213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:29.241934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.242228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.242730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.379415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:29.381799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:29.418074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 832/1625] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:29.419153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:29.465758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 794/1626] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:29.465758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d28c-4916-4f23-8531-38c6926a05b3 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.465758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d28c-4916-4f23-8531-38c6926a05b3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.465758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d28c-4916-4f23-8531-38c6926a05b3 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.501697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 795/1627] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jul 28 01:13:29 2022] GET /identity => generated 270 bytes in 38 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:29.503586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.504533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.504533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.550131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9107f39-2ccf-40f6-8129-d23e70f95c0d service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:29.574157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bedac0-aa52-4c43-a553-74a1f718cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:29.575042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bedac0-aa52-4c43-a553-74a1f718cdf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.577642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bedac0-aa52-4c43-a553-74a1f718cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.578038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bedac0-aa52-4c43-a553-74a1f718cdf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.578882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bedac0-aa52-4c43-a553-74a1f718cdf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:29.584732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bedac0-aa52-4c43-a553-74a1f718cdf6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:29.592151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:29.593598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:29.638989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 796/1628] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jul 28 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:29.641545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:29.653928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 833/1629] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:29 2022] POST /identity/v3/projects => generated 403 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:29.659748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e14402-813c-4594-ada3-4c0d3360173f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.666897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e14402-813c-4594-ada3-4c0d3360173f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.667458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e14402-813c-4594-ada3-4c0d3360173f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.669803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 834/1630] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jul 28 01:13:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:29.670791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e14402-813c-4594-ada3-4c0d3360173f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:29.676169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:29.784308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:29.841619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee8a65-05b5-42d8-bb28-aee2ddbd8124 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:29.889000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e14402-813c-4594-ada3-4c0d3360173f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:29.904597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9fe03-7181-457d-a3d9-db478607602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:29.905420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9fe03-7181-457d-a3d9-db478607602a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.905800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9fe03-7181-457d-a3d9-db478607602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.906184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9fe03-7181-457d-a3d9-db478607602a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.907647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9fe03-7181-457d-a3d9-db478607602a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:29.915038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9fe03-7181-457d-a3d9-db478607602a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:29.970688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 835/1631] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:29 2022] POST /identity/v3/users => generated 308 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:29.972213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d574bd3-4c97-4c80-b2c9-32464de9dcf3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.974665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d574bd3-4c97-4c80-b2c9-32464de9dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.975484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d574bd3-4c97-4c80-b2c9-32464de9dcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:29.992519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:29.993361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:29.993820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:29.994280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.035060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1d574bd3-4c97-4c80-b2c9-32464de9dcf3 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:30.038898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d574bd3-4c97-4c80-b2c9-32464de9dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:30.174465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.176318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.210999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 797/1632] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:29 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:30.211912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:30.236632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c41a91-50d4-47dc-9b06-3e1024c90ad8 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:30.244337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdd802-e7df-40d2-b608-65ca677bcdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:30.245697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdd802-e7df-40d2-b608-65ca677bcdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.246201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdd802-e7df-40d2-b608-65ca677bcdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.246492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdd802-e7df-40d2-b608-65ca677bcdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.246771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 836/1633] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:30.249450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b660fc-037f-4859-bcc4-bf54fa467a7d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.249740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b660fc-037f-4859-bcc4-bf54fa467a7d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.250023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b660fc-037f-4859-bcc4-bf54fa467a7d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.253803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdd802-e7df-40d2-b608-65ca677bcdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.256323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdd802-e7df-40d2-b608-65ca677bcdf9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.263164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 837/1634] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jul 28 01:13:30 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:30.264877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b660fc-037f-4859-bcc4-bf54fa467a7d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:30.271609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 798/1635] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:30.273140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.273543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.273965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.303706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b660fc-037f-4859-bcc4-bf54fa467a7d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:30.304596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:30.306201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:30.315560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:30.316222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.316560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.316838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.331435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 799/1636] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:13:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:30.335086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:30.355212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.356879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.378015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645a4d7-bda4-454e-8486-0535aca57d9f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:30.384932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:30.385830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.386972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.387272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] PATH_INFO: `/v3/projects/a5567fd80c284acaa6ac88e5300684fa/users/311a0db01c5646558c395672d7cf65c7/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.394712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 838/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:30.426814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:30.444139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126176c2-87f4-42e4-b1f8-31e1f7032269 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:30.452266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5567fd80c284acaa6ac88e5300684fa, user_id=311a0db01c5646558c395672d7cf65c7, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.460350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.460939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:30.461607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.461893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.462255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.489103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.491271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.496543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 800/1638] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:30 2022] PUT /identity/v3/projects/a5567fd80c284acaa6ac88e5300684fa/users/311a0db01c5646558c395672d7cf65c7/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:30.503372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:30.551359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 839/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:30.661808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:30.742409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65982be2-0b00-4830-acf2-4ba82abae34d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:30.757405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac959f3-cde4-4e92-93d0-a3d73a47a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:30.758433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac959f3-cde4-4e92-93d0-a3d73a47a8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.758880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac959f3-cde4-4e92-93d0-a3d73a47a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.763357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac959f3-cde4-4e92-93d0-a3d73a47a8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.767539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac959f3-cde4-4e92-93d0-a3d73a47a8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.769452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac959f3-cde4-4e92-93d0-a3d73a47a8aa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.803704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 801/1640] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:30.804772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:30.828526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181eb19-c49c-462f-a8ec-2b8b42fd79d5 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:30.851316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:30.854122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.854418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.854753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] PATH_INFO: `/v3/projects/a5567fd80c284acaa6ac88e5300684fa/users/311a0db01c5646558c395672d7cf65c7/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:30.865943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5567fd80c284acaa6ac88e5300684fa, user_id=311a0db01c5646558c395672d7cf65c7, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:30.875976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:30.918293 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 840/1641] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:30 2022] PUT /identity/v3/projects/a5567fd80c284acaa6ac88e5300684fa/users/311a0db01c5646558c395672d7cf65c7/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:30.921206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:30.922728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:30.924694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:31.001932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] MFA Rules not processed for user `311a0db01c5646558c395672d7cf65c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:31.003489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:31.230261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 802/1642] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:30 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:31.356368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:31.496156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000579e0-a6c9-493f-9d39-5066e5cbc107 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:31.501565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e719-b4b4-4c72-8514-5d215742e1bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:31.502949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e719-b4b4-4c72-8514-5d215742e1bd service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:31.503482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e719-b4b4-4c72-8514-5d215742e1bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:31.505216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e719-b4b4-4c72-8514-5d215742e1bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:31.540504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e719-b4b4-4c72-8514-5d215742e1bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:31.545489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e719-b4b4-4c72-8514-5d215742e1bd service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:31.591121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 841/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:31 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:32.114965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:32.375385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b99b-e834-4b3d-8d75-00e319ca8990 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:32.388363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:32.389681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:32.390193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:32.392497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] PATH_INFO: `/v3/users/302f4cf1c97941a09dd1734ee549825d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:32.421518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=302f4cf1c97941a09dd1734ee549825d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:32.428534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:32.590414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302f4cf1c97941a09dd1734ee549825d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:32.613480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302f4cf1c97941a09dd1734ee549825d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:32.614098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302f4cf1c97941a09dd1734ee549825d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:32.619142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:32.670332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302f4cf1c97941a09dd1734ee549825d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:32.708951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8667020d-99db-40b8-9015-0f2eb90a24f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302f4cf1c97941a09dd1734ee549825d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:32.740465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 803/1644] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:32 2022] DELETE /identity/v3/users/302f4cf1c97941a09dd1734ee549825d => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:32.745536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:32.746087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:32.746353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:32.788551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:32.793290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:32.841692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:32.986924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 842/1645] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:33.002115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:33.002415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:33.002707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:33.024854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:33.028218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:33.117874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] MFA Rules not processed for user `84b16be4f85b440a9e3afea73ffc7871`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:33.125055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:33.212361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee61f3-3e26-40e4-8665-432af429dedc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:33.225684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc932774-5b74-45a4-8592-e493febb478e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:33.248197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932774-5b74-45a4-8592-e493febb478e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:33.249163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932774-5b74-45a4-8592-e493febb478e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:33.250239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc932774-5b74-45a4-8592-e493febb478e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:33.281844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc932774-5b74-45a4-8592-e493febb478e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:33.285244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc932774-5b74-45a4-8592-e493febb478e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:33.324639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 843/1646] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:13:33.329742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 804/1647] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:33.340870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa06381-b19e-4765-aa40-7b6c77e85b8c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:33.341471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa06381-b19e-4765-aa40-7b6c77e85b8c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:33.342027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa06381-b19e-4765-aa40-7b6c77e85b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:33.372311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5aa06381-b19e-4765-aa40-7b6c77e85b8c None None] MFA Rules not processed for user `84b16be4f85b440a9e3afea73ffc7871`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:33.377853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5aa06381-b19e-4765-aa40-7b6c77e85b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:33.399478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 844/1648] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:33.411114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:33.412897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:33.662160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f42f9c-a9d6-49e7-ad42-d7de5b34e911 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:33.684060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa15181-7f06-48cd-b982-9466bb385302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:33.685021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa15181-7f06-48cd-b982-9466bb385302 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:33.685534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa15181-7f06-48cd-b982-9466bb385302 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:33.686118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa15181-7f06-48cd-b982-9466bb385302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:33.709913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa15181-7f06-48cd-b982-9466bb385302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:33.715547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa15181-7f06-48cd-b982-9466bb385302 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:33.742889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 805/1649] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:13:34.932691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd053f-623e-43a5-bd24-4cb6f03c4289 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:34.933731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd053f-623e-43a5-bd24-4cb6f03c4289 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:34.934258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd053f-623e-43a5-bd24-4cb6f03c4289 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:34.936685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 845/1650] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jul 28 01:13:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:34.943729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:34.944831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:34.945418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.025179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:35.026453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:35.035734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 806/1651] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:13:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 28 01:13:35.059267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd053f-623e-43a5-bd24-4cb6f03c4289 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:35.084545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd053f-623e-43a5-bd24-4cb6f03c4289 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:35.107026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:35.107026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:35.107026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:35.107026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.140966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:35.143778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:35.162211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:35.163969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:35.179653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 846/1652] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:35.195400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c83c5c-d3b4-4c91-b9e7-7f654e8617da None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:35.207744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:35.209554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:35.209863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:35.211397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] PATH_INFO: `/v3/projects/74152caac2cf4f8abdd7855f775ec5ce` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.214478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=74152caac2cf4f8abdd7855f775ec5ce)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:35.218120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:13:35.218120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: warnings.warn(msg) Jul 28 01:13:35.218120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: Jul 28 01:13:35.224386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:35.302212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:35.304368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:35.307299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:35.350024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0c97d-bb66-44d8-8d00-f00162f07833 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:35.395188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74152caac2cf4f8abdd7855f775ec5ce', 'request_id': 'req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.415612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74152caac2cf4f8abdd7855f775ec5ce', 'request_id': 'req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.438501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74152caac2cf4f8abdd7855f775ec5ce', 'request_id': 'req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.454893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:35.455583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:35.455886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:35.456143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] PATH_INFO: `/v3/users/3fcfb45bc3bc48ec9c6cdd33d18b66b0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.457416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fcfb45bc3bc48ec9c6cdd33d18b66b0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:35.460289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 01:13:35.460289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: warnings.warn(msg) Jul 28 01:13:35.460289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: Jul 28 01:13:35.461887 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:35.489979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] The token cache is being invalidate because project 74152caac2cf4f8abdd7855f775ec5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:35.490651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.492567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.500914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 807/1653] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:35 2022] DELETE /identity/v3/projects/74152caac2cf4f8abdd7855f775ec5ce => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:35.502616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:35.504643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:35.524255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135a6eb-2091-4ed7-bc47-d0f6c48ea42a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:35.531023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:35.532271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:35.533054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:35.533358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] PATH_INFO: `/v3/users/311a0db01c5646558c395672d7cf65c7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.535307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=311a0db01c5646558c395672d7cf65c7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:35.537621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:35.557034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:35.561848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fcfb45bc3bc48ec9c6cdd33d18b66b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.563113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fcfb45bc3bc48ec9c6cdd33d18b66b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.564838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fcfb45bc3bc48ec9c6cdd33d18b66b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.580305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fcfb45bc3bc48ec9c6cdd33d18b66b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.616322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fcfb45bc3bc48ec9c6cdd33d18b66b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.648345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '311a0db01c5646558c395672d7cf65c7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.659299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 847/1654] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:35 2022] DELETE /identity/v3/users/3fcfb45bc3bc48ec9c6cdd33d18b66b0 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:13:35.661806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:35.662985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '311a0db01c5646558c395672d7cf65c7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.664051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '311a0db01c5646558c395672d7cf65c7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.664988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:35.681383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '311a0db01c5646558c395672d7cf65c7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.696410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b1487112-a588-403a-9b56-26ca5be8b442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '311a0db01c5646558c395672d7cf65c7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:35.708623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 808/1655] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:35 2022] DELETE /identity/v3/users/311a0db01c5646558c395672d7cf65c7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:35.710869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:35.711371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:35.711843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.766034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:35.770399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:35.781241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890928fb-6b19-419e-b461-c0f6c877de4e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:35.910971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:35.911184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 809/1656] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:35.911265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:35.911265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:35.911265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] PATH_INFO: `/v3/users/59c4bd188120450fa69be750c9308521` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:35.913882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c4bd188120450fa69be750c9308521)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:35.915935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:35.943369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:35.945162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:36.060629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:36.070155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c4bd188120450fa69be750c9308521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:36.070809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c4bd188120450fa69be750c9308521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:36.073267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c4bd188120450fa69be750c9308521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:36.111374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c4bd188120450fa69be750c9308521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:36.119971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ffaf1-c4d9-42ec-abf2-602d4f2136f4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:36.124325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:36.125125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:36.125415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:36.125701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:36.127986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:36.147455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:36.154227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80563a-0525-4a72-95b2-032543ee25a4 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:36.168754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:36.172080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db70786a-b6a2-42fe-918b-76c651d40614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c4bd188120450fa69be750c9308521'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:36.188587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 810/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:13:36.205570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 848/1658] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:35 2022] DELETE /identity/v3/users/59c4bd188120450fa69be750c9308521 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:36.210558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ff82f-3f2a-4f16-9e75-e3956d8c2ce4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:36.215352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ff82f-3f2a-4f16-9e75-e3956d8c2ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:36.215671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ff82f-3f2a-4f16-9e75-e3956d8c2ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:36.251234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6d4ff82f-3f2a-4f16-9e75-e3956d8c2ce4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:36.252621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6d4ff82f-3f2a-4f16-9e75-e3956d8c2ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:36.364150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 811/1659] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:36.380338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afea50-cc10-423c-a82e-ee606010aa41 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:36.381246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afea50-cc10-423c-a82e-ee606010aa41 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:36.381595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afea50-cc10-423c-a82e-ee606010aa41 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:36.383120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 849/1660] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jul 28 01:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:36.395087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:36.395814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:36.396111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:36.399201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afea50-cc10-423c-a82e-ee606010aa41 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:36.403723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d2afea50-cc10-423c-a82e-ee606010aa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:36.481566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:36.488596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:36.630333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afea50-cc10-423c-a82e-ee606010aa41 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:36.640599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:36.643822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:36.643822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:36.643822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:36.644216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 812/1661] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jul 28 01:13:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:36.654405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:36.672064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c526cc-29a8-475b-a8fd-0e64a14fa505 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:36.679984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:36.685176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:36.690269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:36.690868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:36.691144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:36.691425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:36.693363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:36.695774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 850/1662] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:36.708973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:36.712531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:36.737115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 813/1663] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:37.789908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:37.791791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:37.817107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60aa84-53cf-4d22-a05d-52955f1c40b8 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:37.829150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:37.829873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:37.830259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:37.830765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] PATH_INFO: `/v3/projects/a5567fd80c284acaa6ac88e5300684fa` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:37.832459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5567fd80c284acaa6ac88e5300684fa)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:37.844918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:37.890472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:37.890472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:37.926146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f68e57-e405-4aa5-b032-39b9e04752da service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:37.940165 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:37.941379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:37.945072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:37.945421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] PATH_INFO: `/v3/projects/8fe8407ac0834896b8693848d6c023d8` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:37.956054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe8407ac0834896b8693848d6c023d8)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:37.971091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:38.016786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5567fd80c284acaa6ac88e5300684fa', 'request_id': 'req-5b16b2ee-8289-47e3-a7a6-8051e34379dc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.028771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5567fd80c284acaa6ac88e5300684fa', 'request_id': 'req-5b16b2ee-8289-47e3-a7a6-8051e34379dc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.045330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5567fd80c284acaa6ac88e5300684fa', 'request_id': 'req-5b16b2ee-8289-47e3-a7a6-8051e34379dc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.067204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] The token cache is being invalidate because project a5567fd80c284acaa6ac88e5300684fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:38.068925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.069835 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5b16b2ee-8289-47e3-a7a6-8051e34379dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.076247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 851/1664] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:37 2022] DELETE /identity/v3/projects/a5567fd80c284acaa6ac88e5300684fa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:38.081241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c5d0-c2bb-48b7-a50c-f492c0431dbf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.081538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c5d0-c2bb-48b7-a50c-f492c0431dbf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.082054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11c5d0-c2bb-48b7-a50c-f492c0431dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.089801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:38.094416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe8407ac0834896b8693848d6c023d8', 'request_id': 'req-f586bca5-963b-42cd-af3a-381221329dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.105794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe8407ac0834896b8693848d6c023d8', 'request_id': 'req-f586bca5-963b-42cd-af3a-381221329dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.115119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe8407ac0834896b8693848d6c023d8', 'request_id': 'req-f586bca5-963b-42cd-af3a-381221329dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.137314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] The token cache is being invalidate because project 8fe8407ac0834896b8693848d6c023d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:38.137745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.138694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f586bca5-963b-42cd-af3a-381221329dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:38.142912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 814/1665] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:37 2022] DELETE /identity/v3/projects/8fe8407ac0834896b8693848d6c023d8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:38.143086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ea11c5d0-c2bb-48b7-a50c-f492c0431dbf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:38.147919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a04a-ae85-4e41-8495-b2f79773dab7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.148424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a04a-ae85-4e41-8495-b2f79773dab7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.148844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795a04a-ae85-4e41-8495-b2f79773dab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.154496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ea11c5d0-c2bb-48b7-a50c-f492c0431dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:38.212051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1795a04a-ae85-4e41-8495-b2f79773dab7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:38.213327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1795a04a-ae85-4e41-8495-b2f79773dab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:38.311925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 852/1666] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:38.319553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.319858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.320131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.370091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:38.375384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:38.391079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 815/1667] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:38.400016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.402075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.402908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.415792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 853/1668] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:13:38.420250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:38.444732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a6867-43ab-40dd-9ce1-2cd4a32c936f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:38.526698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:38.530719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:38.532658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.534992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:38.535581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.536101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.540485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:38.541491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:38.544904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:38.561758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:38.583712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 816/1669] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:38.585917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:38.616683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 854/1670] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:38.621186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:38.629454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a5e1d-ea81-45cc-a58c-edeebf0fad94 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:38.649599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:38.649981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.650282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.650952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.651661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:38.651966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:38.664744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffb9a8-8e80-4770-99fb-ad300603bfa5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:38.665906 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:38.679808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:38.680595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.680886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.681189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.681997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:38.692983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:38.723269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 817/1671] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:38.730289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:38.791263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2f166-91da-4c52-8256-eb7645ac2989 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:38.805753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:38.806544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:38.806878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:38.807158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:38.816200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:38.822183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:38.830368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 855/1672] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:38.839907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:39.011095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 818/1673] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/projects => generated 363 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:39.011914 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:39.019804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:39.109558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4468dbc-2c43-46cf-826b-b44182b4aea8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:39.229873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea1455-59c1-431e-b2ba-69308f02f4d5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:39.252018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:39.252018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:39.252018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:39.254090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] PATH_INFO: `/v3/projects/3f9548d25b094b9a9d5418410c044445` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:39.257616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9548d25b094b9a9d5418410c044445)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:39.261200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:39.340177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:39.340658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:39.343189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:39.343589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:39.344409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:39.346533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:39.427495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 856/1674] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:38 2022] POST /identity/v3/users => generated 290 bytes in 593 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:13:39.428619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:39.429218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:39.480182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9548d25b094b9a9d5418410c044445', 'request_id': 'req-082a3d48-867b-4648-af7c-4de4fd1cc856'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:39.480778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb712452-5fed-49c2-ba3d-3b9dbc7ed125 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:39.527253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9548d25b094b9a9d5418410c044445', 'request_id': 'req-082a3d48-867b-4648-af7c-4de4fd1cc856'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:39.556947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9548d25b094b9a9d5418410c044445', 'request_id': 'req-082a3d48-867b-4648-af7c-4de4fd1cc856'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:39.623350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] The token cache is being invalidate because project 3f9548d25b094b9a9d5418410c044445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:39.625019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:39.628781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:39.638760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 819/1675] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:39 2022] DELETE /identity/v3/projects/3f9548d25b094b9a9d5418410c044445 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:39.641632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:39.649586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:39.742898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:39.743808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:39.744319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:39.744883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:39.746423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:39.753191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:39.800323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:39.812211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 857/1676] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:39 2022] POST /identity/v3/users => generated 282 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:39.813423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:39.820863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:39.844390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b22892-aa45-4196-beae-1f799f654acc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:39.871052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:39.871970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:39.872266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:39.872555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] PATH_INFO: `/v3/users/490388172bdd4b9ea44267169746856c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:39.874224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490388172bdd4b9ea44267169746856c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:39.877506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:39.959493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a3d48-867b-4648-af7c-4de4fd1cc856 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:39.987854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:39.989124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:39.989646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:39.990124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:39.991448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:39.999771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:40.039548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 820/1677] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:40.042442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:40.044557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:40.061462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490388172bdd4b9ea44267169746856c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:40.063044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490388172bdd4b9ea44267169746856c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:40.065391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490388172bdd4b9ea44267169746856c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:40.069183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c16ae2-73c7-4a8e-9dae-1abb28e59ae3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:40.091611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490388172bdd4b9ea44267169746856c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:40.094810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:40.099549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.100964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.101501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.103368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:40.109285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:40.134343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490388172bdd4b9ea44267169746856c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:40.140064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 821/1678] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:40.145174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:40.152658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:40.174796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 858/1679] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:39 2022] DELETE /identity/v3/users/490388172bdd4b9ea44267169746856c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:40.179196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:40.182602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:40.193878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f536-8596-4c75-bd90-6bf776fcef2b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:40.381546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:40.382941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.383223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.383511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.384238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:40.391961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:40.421235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3811ce-839e-494c-99de-86e8ed97b123 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:40.426815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 822/1680] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:40.431184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:40.456282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:40.457113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.457523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.457628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666362d2-dd0d-4439-a967-e270bcd4c2cf admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:40.457976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.458809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:40.460811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:40.478615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63617281-8bed-4536-b6b6-c36272e2186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:40.479927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617281-8bed-4536-b6b6-c36272e2186a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.479927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617281-8bed-4536-b6b6-c36272e2186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.479927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617281-8bed-4536-b6b6-c36272e2186a admin admin] PATH_INFO: `/v3/projects/6336fa83cba644809833f46e4f259be3/users/9fb6e2a043c1414a9fb4733505604595/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.500332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 859/1681] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:40.501771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:40.532854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63617281-8bed-4536-b6b6-c36272e2186a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6336fa83cba644809833f46e4f259be3, user_id=9fb6e2a043c1414a9fb4733505604595, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:40.537675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63617281-8bed-4536-b6b6-c36272e2186a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:40.538456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3088a-6d45-4366-8924-814ed985407b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:40.556694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d42f53-d736-43f7-a40a-6753aae3bbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:40.557599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42f53-d736-43f7-a40a-6753aae3bbed admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.558087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42f53-d736-43f7-a40a-6753aae3bbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.558474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42f53-d736-43f7-a40a-6753aae3bbed admin admin] PATH_INFO: `/v3/projects/e0b875c08c024b30bd641438f5d4e56b/users/45a0b1bf72d342898fca4250e84cff39/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.569389 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 823/1682] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:40 2022] PUT /identity/v3/projects/6336fa83cba644809833f46e4f259be3/users/9fb6e2a043c1414a9fb4733505604595/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:13:40.580948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515f0b-ce97-45bf-98b1-f8d22f181059 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.581262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515f0b-ce97-45bf-98b1-f8d22f181059 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.581483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515f0b-ce97-45bf-98b1-f8d22f181059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.601338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d42f53-d736-43f7-a40a-6753aae3bbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b875c08c024b30bd641438f5d4e56b, user_id=45a0b1bf72d342898fca4250e84cff39, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:40.607138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d42f53-d736-43f7-a40a-6753aae3bbed admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:40.643656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b1515f0b-ce97-45bf-98b1-f8d22f181059 None None] MFA Rules not processed for user `9fb6e2a043c1414a9fb4733505604595`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:40.644942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1515f0b-ce97-45bf-98b1-f8d22f181059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:40.662649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 860/1683] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:40 2022] PUT /identity/v3/projects/e0b875c08c024b30bd641438f5d4e56b/users/45a0b1bf72d342898fca4250e84cff39/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:40.662996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.663412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.663813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.668881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:40.774183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:40.775361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:40.809887 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:40.818573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 824/1684] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:13:40.820409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f625-33ef-44da-9852-cb3591de163d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:40.820409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f625-33ef-44da-9852-cb3591de163d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:40.820764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223f625-33ef-44da-9852-cb3591de163d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:40.893427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1223f625-33ef-44da-9852-cb3591de163d None None] MFA Rules not processed for user `45a0b1bf72d342898fca4250e84cff39`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:40.897904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1223f625-33ef-44da-9852-cb3591de163d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:40.956345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 861/1685] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:13:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:13:40.957528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.126219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 825/1686] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:40 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:41.131727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223f625-33ef-44da-9852-cb3591de163d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.149866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223f625-33ef-44da-9852-cb3591de163d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:41.158259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417925b-319c-413b-a923-b6f2b9d7ef1d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:41.159559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:41.167234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:41.167617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:41.167911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:41.168780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:41.178430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:41.180937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:41.181436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:41.181839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:41.182221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:41.191860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:41.197241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:41.261435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 826/1687] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:41 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:41.261435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.261863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:41.359379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 862/1688] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:40 2022] POST /identity/v3/projects => generated 381 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:41.360350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.509121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:41.522809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c3a80-4a4e-4b1e-9101-cc4f08cacae9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:41.528376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f97c6-4a71-4155-beb9-bd22128ed45f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:41.530477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:41.531185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:41.531469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:41.531758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:41.532966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:41.537865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:41.604492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 863/1689] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:41 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:41.608580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.615926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:41.667039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6cb3c-9e4b-44f8-86d1-1a395f72b721 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:41.678424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:41.679595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:41.679886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:41.680346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] PATH_INFO: `/v3/users/4c9c7d609b4a473193d7ed97c4b7ffd1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:41.682130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9c7d609b4a473193d7ed97c4b7ffd1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:41.685403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:41.691400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:41.692480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:41.692910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:41.693356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:41.715778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:41.795737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9c7d609b4a473193d7ed97c4b7ffd1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:41.798288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9c7d609b4a473193d7ed97c4b7ffd1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:41.801827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9c7d609b4a473193d7ed97c4b7ffd1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:41.834132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9c7d609b4a473193d7ed97c4b7ffd1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:41.838073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:41.844772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:41.891678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 827/1690] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:13:41.892741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.893882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9c7d609b4a473193d7ed97c4b7ffd1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:41.931976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f7349-320e-4fe9-8172-fe1603b4039b service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:41.933174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 864/1691] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:41 2022] DELETE /identity/v3/users/4c9c7d609b4a473193d7ed97c4b7ffd1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:41.935020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:41.943148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:42.207847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf841b-e8c5-4e79-a49c-c959fd3a6a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:42.209626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf841b-e8c5-4e79-a49c-c959fd3a6a20 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:42.209952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf841b-e8c5-4e79-a49c-c959fd3a6a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:42.210237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf841b-e8c5-4e79-a49c-c959fd3a6a20 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:42.210987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf841b-e8c5-4e79-a49c-c959fd3a6a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:42.222596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf841b-e8c5-4e79-a49c-c959fd3a6a20 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:42.264750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b2794-2cad-4092-b3b4-83ff0b2c863c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:42.281314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:42.286875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:42.289721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:42.290046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:42.300563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:42.305043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:42.315532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 828/1692] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:41 2022] POST /identity/v3/users => generated 297 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:42.321820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:42.328034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:42.332155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:42.363428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 865/1693] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:42.364774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:42.374611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:42.422572 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:42.426004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:42.440937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4cc62-8500-412f-91d4-0a4e600922fe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:42.490792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:42.494478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:42.495645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 829/1694] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:42.496537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:42.497450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:42.499068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:42.500466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:42.506124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:42.542040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a8b3c-a38c-4eba-b4f9-a885bf2442cd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:42.545322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 866/1695] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:42.549363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:42.554228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:42.560111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:42.562708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:42.564459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:42.564961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] PATH_INFO: `/v3/projects/d517899c0f604866b1e0229dcd42818d/users/416a26ca40234b51acb7763dcf7de372/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:42.581446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ddcbb-c70e-48e4-98f8-ae1e1ce9d576 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:42.599575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:42.602164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:42.602648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:42.604124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] PATH_INFO: `/v3/projects/4ab8a4d21b014c59874e3a80f3e5d36e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:42.609588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab8a4d21b014c59874e3a80f3e5d36e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:42.614283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:42.640416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d517899c0f604866b1e0229dcd42818d, user_id=416a26ca40234b51acb7763dcf7de372, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:42.646485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:42.678048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:42.698471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 830/1696] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:42 2022] PUT /identity/v3/projects/d517899c0f604866b1e0229dcd42818d/users/416a26ca40234b51acb7763dcf7de372/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:42.699781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:42.851103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab8a4d21b014c59874e3a80f3e5d36e', 'request_id': 'req-653e9b29-01ad-41e6-811f-c7c2a24e9408'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:42.893390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab8a4d21b014c59874e3a80f3e5d36e', 'request_id': 'req-653e9b29-01ad-41e6-811f-c7c2a24e9408'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:42.918648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add12fd3-12fa-4271-b4a2-9167d2a848e4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:42.919332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab8a4d21b014c59874e3a80f3e5d36e', 'request_id': 'req-653e9b29-01ad-41e6-811f-c7c2a24e9408'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:42.948057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] The token cache is being invalidate because project 4ab8a4d21b014c59874e3a80f3e5d36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:42.953522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:42.955315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:42.962317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 867/1697] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:42 2022] DELETE /identity/v3/projects/4ab8a4d21b014c59874e3a80f3e5d36e => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:42.963469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:42.969596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:42.999862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:43.086705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.087461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.087744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.089362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] PATH_INFO: `/v3/projects/e690da75cdd7412cb975c4a9799d9fce/users/24f684c7588e4c10933ba799a8cdc995/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.124565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e9b29-01ad-41e6-811f-c7c2a24e9408 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:43.128525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e690da75cdd7412cb975c4a9799d9fce, user_id=24f684c7588e4c10933ba799a8cdc995, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.133035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.138324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.138993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.139289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.139558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.143020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:43.163096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 831/1698] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:42 2022] PUT /identity/v3/projects/e690da75cdd7412cb975c4a9799d9fce/users/24f684c7588e4c10933ba799a8cdc995/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:43.163845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:43.166521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:43.173238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.175464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e0a1-1abb-482e-b30a-160f10d2e0e4 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.203447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 868/1699] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:43.207438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.207830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.208297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.289520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:43.293324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:43.321353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 869/1700] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:43.322402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:43.327540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:43.342694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad15b46-4919-4167-856a-e87363b2ee52 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:43.356868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb3d42-1964-47bc-bda2-262cdaf196e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.358406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d230-5f3c-4386-b0a0-b5e93eb7476e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:43.360620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3d42-1964-47bc-bda2-262cdaf196e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.360911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3d42-1964-47bc-bda2-262cdaf196e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.361200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3d42-1964-47bc-bda2-262cdaf196e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.366887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb3d42-1964-47bc-bda2-262cdaf196e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.371126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb3d42-1964-47bc-bda2-262cdaf196e5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.380833 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.381584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.387971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.388569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.391344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.393162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.410977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 832/1701] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:43.410977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.410977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.417438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.437425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 870/1702] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:43.441313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:43.482396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010227ff-d35a-44c4-979c-b1e7e913dcba admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:43.498317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.500862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.501702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:43.502155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.502460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] PATH_INFO: `/v3/projects/d517899c0f604866b1e0229dcd42818d/users/416a26ca40234b51acb7763dcf7de372/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.505880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:43.519046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d517899c0f604866b1e0229dcd42818d, user_id=416a26ca40234b51acb7763dcf7de372, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.525741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.531678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 833/1703] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:43.532815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:43.557900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 871/1704] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:43 2022] PUT /identity/v3/projects/d517899c0f604866b1e0229dcd42818d/users/416a26ca40234b51acb7763dcf7de372/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:43.567643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:43.577113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e95c3-d416-4a3c-82b4-ec61ccf0fd2d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:43.639084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:43.850339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa8ca2-6037-4b2c-a686-69fb400d9f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.852356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8ca2-6037-4b2c-a686-69fb400d9f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.854365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622599b4-6763-411d-b387-4337763a67b3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:43.855430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8ca2-6037-4b2c-a686-69fb400d9f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.855741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8ca2-6037-4b2c-a686-69fb400d9f0e admin admin] PATH_INFO: `/v3/projects/e690da75cdd7412cb975c4a9799d9fce/users/24f684c7588e4c10933ba799a8cdc995/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.868306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa8ca2-6037-4b2c-a686-69fb400d9f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e690da75cdd7412cb975c4a9799d9fce, user_id=24f684c7588e4c10933ba799a8cdc995, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.873222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa8ca2-6037-4b2c-a686-69fb400d9f0e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.876485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:43.879133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.882625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.883254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.887557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:43.888920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:43.896182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72e0ff-bfb2-4d1f-ac9e-6cab11832da9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:43.918958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 834/1705] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:43 2022] PUT /identity/v3/projects/e690da75cdd7412cb975c4a9799d9fce/users/24f684c7588e4c10933ba799a8cdc995/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:43.937268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.938014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.938550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:43.958470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 872/1706] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:43.978759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:43.979107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:43.979384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:44.021336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] MFA Rules not processed for user `416a26ca40234b51acb7763dcf7de372`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:44.024150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:44.059373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] MFA Rules not processed for user `24f684c7588e4c10933ba799a8cdc995`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:44.060886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:44.217336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:44.224984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 873/1707] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:44.226271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:44.293659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 835/1708] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:44.293659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:44.294518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:44.398186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1778c-7e7c-400f-a795-4a98e0354f38 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:44.410822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:44.411238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:44.411573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:44.411833 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:44.412540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:44.416409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:44.490743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fc98d-c7b5-48a3-9d19-9040e06f357e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:44.518995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:44.519688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 874/1709] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:44 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:44.519909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:44.520267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:44.520555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:44.524741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:44.535526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:44.559918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:44.564576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:44.612827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 836/1710] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:44.613755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:44.664890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8e14c-7296-4854-ac1e-1e6644b4d621 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:44.730977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07d8f-0792-4d9e-a0e2-c1e002dc874e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:44.741195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:44.744238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:44.744621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:44.744909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:44.853824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:44.911378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:44.912349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:44.913019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:44.913447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:44.914305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:44.922984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:44.984259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 837/1711] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:44 2022] POST /identity/v3/users => generated 294 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:44.985997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:45.020028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be30a-538e-4eb9-80e2-17267eea9acd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:45.037677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:45.038428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:45.038714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:45.039016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:45.042873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:45.047234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:45.047661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:45.091185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 875/1712] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:45.098467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:45.132068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e348-3bdc-4f8a-9fff-a57b6c645710 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:45.155227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:45.156029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:45.156306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:45.156590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:45.157366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:45.166028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:45.207394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 876/1713] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:45.210643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:45.235968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c89af-e685-43cf-b561-f3ae05e8b2b0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:45.251730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:45.253643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:45.254104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:45.254467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:45.256156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:45.259311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:45.290394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:45.293031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 877/1714] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:45.297659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:45.306446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be891bde-9802-4037-b85b-ba266515dab4 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:45.337839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e03d93-0658-4f38-acd9-dc5e36605dbe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:45.342104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 838/1715] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:45.344122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:45.345192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:45.345488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:45.345772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] PATH_INFO: `/v3/projects/1840ddc234704ee18e9ab9b927ec10bf/users/edb144a2af2a47469600c8438c63eeeb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:45.374635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:45.384281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1840ddc234704ee18e9ab9b927ec10bf, user_id=edb144a2af2a47469600c8438c63eeeb, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:45.392851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:45.450107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 878/1716] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:45 2022] PUT /identity/v3/projects/1840ddc234704ee18e9ab9b927ec10bf/users/edb144a2af2a47469600c8438c63eeeb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:45.460727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:45.461094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:45.461435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:45.562742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] MFA Rules not processed for user `edb144a2af2a47469600c8438c63eeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:45.564394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:45.822467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 839/1717] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:45.824726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:46.132760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a66ac-6a9d-436b-8762-ad4e10cf1adc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:46.175594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:46.180126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:46.182652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:46.183002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:46.183757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:46.188534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:46.353597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 879/1718] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:45 2022] POST /identity/v3/projects => generated 387 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:46.356463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:46.358470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:46.370779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:46.428099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6f7c4-0b59-4dee-8357-e9077f6d377d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:46.671093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:46.676716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:46.677403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:46.678588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] PATH_INFO: `/v3/projects/ba821829826b4d499eb669af047a14d9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:46.692186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba821829826b4d499eb669af047a14d9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:46.710594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:46.716674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:46.797064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ef701-59cc-4dd1-bdb8-1bf7f2eb9be8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:46.824892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:46.825445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:46.825734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:46.826014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:46.826713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:46.848126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:46.872788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba821829826b4d499eb669af047a14d9', 'request_id': 'req-eda98fb7-4782-4cad-8f91-74dc13a54b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:46.903551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba821829826b4d499eb669af047a14d9', 'request_id': 'req-eda98fb7-4782-4cad-8f91-74dc13a54b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:46.922301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba821829826b4d499eb669af047a14d9', 'request_id': 'req-eda98fb7-4782-4cad-8f91-74dc13a54b5c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:46.939415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 880/1719] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:46 2022] POST /identity/v3/users => generated 300 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:46.940452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:46.965963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] The token cache is being invalidate because project ba821829826b4d499eb669af047a14d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:46.966270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:46.970529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:46.977587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 840/1720] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:46 2022] DELETE /identity/v3/projects/ba821829826b4d499eb669af047a14d9 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:46.984919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:46.989817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:47.007173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8882-2547-452a-9588-b584d04b39bc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:47.244960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:47.251554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:47.253180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:47.253602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] PATH_INFO: `/v3/users/45a0b1bf72d342898fca4250e84cff39` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:47.262623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a0b1bf72d342898fca4250e84cff39)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:47.268620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:47.275701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda98fb7-4782-4cad-8f91-74dc13a54b5c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:47.293600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28793c-a9e4-473b-ab2b-eb70052c8320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:47.294301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28793c-a9e4-473b-ab2b-eb70052c8320 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:47.294585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28793c-a9e4-473b-ab2b-eb70052c8320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:47.295035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28793c-a9e4-473b-ab2b-eb70052c8320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:47.295620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28793c-a9e4-473b-ab2b-eb70052c8320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:47.298644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:47.313487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28793c-a9e4-473b-ab2b-eb70052c8320 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:47.334159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 841/1721] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:47.358372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:47.358760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:47.358952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:47.438489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:47.444894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a0b1bf72d342898fca4250e84cff39'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:47.447464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a0b1bf72d342898fca4250e84cff39'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:47.451081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:47.456372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a0b1bf72d342898fca4250e84cff39'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:47.486465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a0b1bf72d342898fca4250e84cff39'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:47.501385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 842/1722] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:47.512549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:47.519502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:47.520495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5325f7f7-7fba-43cc-bdcb-ac3895d90113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a0b1bf72d342898fca4250e84cff39'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:47.554427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 881/1723] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:46 2022] DELETE /identity/v3/users/45a0b1bf72d342898fca4250e84cff39 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:47.556600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2b05f-cf93-4fb1-b2b5-8eabcecfa5fb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:47.556747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2b05f-cf93-4fb1-b2b5-8eabcecfa5fb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:47.557981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2b05f-cf93-4fb1-b2b5-8eabcecfa5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:47.591520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac892a0-b4bd-44a5-8feb-ca1aa6674490 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:47.620131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-96f2b05f-cf93-4fb1-b2b5-8eabcecfa5fb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:47.622931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-96f2b05f-cf93-4fb1-b2b5-8eabcecfa5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:47.804837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:47.818288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:47.823390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:47.823926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] PATH_INFO: `/v3/projects/455c7947fbb94a00b9c24eb8bf305a75/users/abdb989a0bb147758148f43251b71989/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:47.851928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 882/1724] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:47.865264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3aec7-68aa-45ec-9715-4ff3b626c40b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:47.866148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3aec7-68aa-45ec-9715-4ff3b626c40b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:47.866915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3aec7-68aa-45ec-9715-4ff3b626c40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:47.872060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=455c7947fbb94a00b9c24eb8bf305a75, user_id=abdb989a0bb147758148f43251b71989, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:47.874938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:47.910638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 843/1725] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:47 2022] PUT /identity/v3/projects/455c7947fbb94a00b9c24eb8bf305a75/users/abdb989a0bb147758148f43251b71989/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:47.911698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:47.919930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:48.009275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bdb3aec7-68aa-45ec-9715-4ff3b626c40b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:48.016379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bdb3aec7-68aa-45ec-9715-4ff3b626c40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:48.161256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2f422-14b6-4016-a4aa-ab5fccacaf6c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:48.172612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:48.173026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.173306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.173592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.174471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:48.174642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:48.184120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:48.235293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 844/1726] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:48.236361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:48.294120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649cc69-5789-4353-9efd-16f45b902e34 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:48.311969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:48.312647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.312946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.313215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.313945 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:48.316491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:48.359179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 883/1727] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:48.359179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.359179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.359179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.361068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 845/1728] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:48.366142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:48.385872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6bf7-a3ce-426f-94f0-36175e9423e7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:48.402241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:48.404326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.405740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:48.406037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.406363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.407427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:48.412879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:48.412879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:48.469125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:48.513264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 846/1729] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:48 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:48.514469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:48.607034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 884/1730] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jul 28 01:13:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:48.608032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:48.610190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:48.671585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b0764-677b-48c3-b9ad-0b406f96c3d2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:48.677528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed8f41-5ced-4fca-9499-dcfda277370d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:48.678459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:48.679637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.681802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.682260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] PATH_INFO: `/v3/projects/455c7947fbb94a00b9c24eb8bf305a75/users/abdb989a0bb147758148f43251b71989/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.690905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455c7947fbb94a00b9c24eb8bf305a75, user_id=abdb989a0bb147758148f43251b71989, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:48.698222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:48.751663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 847/1731] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:48 2022] PUT /identity/v3/projects/455c7947fbb94a00b9c24eb8bf305a75/users/abdb989a0bb147758148f43251b71989/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:48.753240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:48.908793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:48.919571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049a43-20f2-4826-93fc-afc894788c89 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:48.942287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:48.942943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.943261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.943509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.944295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:48.946469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:48.948032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d980-d9bf-4e68-aeac-b842708a5e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:48.948801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d980-d9bf-4e68-aeac-b842708a5e13 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:48.949016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d980-d9bf-4e68-aeac-b842708a5e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:48.949295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d980-d9bf-4e68-aeac-b842708a5e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:48.988401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d980-d9bf-4e68-aeac-b842708a5e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:48.989925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d980-d9bf-4e68-aeac-b842708a5e13 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.034764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 848/1732] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:48 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:49.035814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:49.036550 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 885/1733] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:49.038043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.038325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.039329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.042790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:49.116978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] MFA Rules not processed for user `abdb989a0bb147758148f43251b71989`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:49.118246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:49.284893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 886/1734] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:49.284893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:49.298878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d9bf6-0b22-4bd8-8ca3-0b8016b834f3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:49.307688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:49.308356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.308639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.308913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.309899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63106f-9ff3-4311-ab2f-8063a6c64599 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:49.318960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:49.319884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.321521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.321843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.322596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:49.324369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.333826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:49.335988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.360797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 887/1735] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:49.362695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:49.364972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:49.376510 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 849/1736] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:49.377290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:49.436154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5fba5-93c1-4cc8-a0fb-921643daa5c3 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:49.455374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:49.456059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.456373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.456816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.461669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:49.472802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.521817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe7dcb-817d-4f71-a60d-7722ba3c1a81 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:49.527980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:49.529077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.529421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.529688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.533296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 850/1737] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:49.540374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:49.542364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:49.548233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:49.550233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.574602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 888/1738] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:49.592360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953770-ab37-4b78-90e9-94c0a4e29357 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:49.598163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd61e4-c9cd-47d1-9a81-668ca2e2778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:49.598998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd61e4-c9cd-47d1-9a81-668ca2e2778e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.599388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd61e4-c9cd-47d1-9a81-668ca2e2778e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.599552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd61e4-c9cd-47d1-9a81-668ca2e2778e admin admin] PATH_INFO: `/v3/projects/6b0a23a62635438ba00b3249ca7e7894/users/ef46d65717de42e49871e4431f6fa028/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.631714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd61e4-c9cd-47d1-9a81-668ca2e2778e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0a23a62635438ba00b3249ca7e7894, user_id=ef46d65717de42e49871e4431f6fa028, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:49.634971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd61e4-c9cd-47d1-9a81-668ca2e2778e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.646427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:49.667655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c47c9-42af-4411-9d78-4f9e8cccf170 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:49.689968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 851/1739] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:49 2022] PUT /identity/v3/projects/6b0a23a62635438ba00b3249ca7e7894/users/ef46d65717de42e49871e4431f6fa028/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:49.692988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.693407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.693872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.746800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] MFA Rules not processed for user `ef46d65717de42e49871e4431f6fa028`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:49.748075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:49.867845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:49.869491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:49.871260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:49.872391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] PATH_INFO: `/v3/users/edb144a2af2a47469600c8438c63eeeb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:49.875861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb144a2af2a47469600c8438c63eeeb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:49.881581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:49.990127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 852/1740] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:49.994585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:50.017303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d5b71-76c7-455d-91e4-108cf4f85a1a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:50.029386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:50.030181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:50.030473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:50.030749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:50.031664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:50.033492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:50.083923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb144a2af2a47469600c8438c63eeeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:50.088089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb144a2af2a47469600c8438c63eeeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:50.095527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb144a2af2a47469600c8438c63eeeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:50.118707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb144a2af2a47469600c8438c63eeeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:50.137905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 853/1741] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:49 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:50.145830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:50.148918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:50.164041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7e67678-427e-4bdf-9df9-b70281640313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb144a2af2a47469600c8438c63eeeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:50.205920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 889/1742] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:49 2022] DELETE /identity/v3/users/edb144a2af2a47469600c8438c63eeeb => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:50.207692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:50.207984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:50.208264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:50.223515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:50.269589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:50.272617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:50.319370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd933d5-0f95-4f91-9fad-c450188aabaa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:50.454167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 890/1743] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:50.479579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:50.480273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:50.480561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:50.480846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:50.481561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:50.483639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:50.534072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 854/1744] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:50 2022] POST /identity/v3/users => generated 289 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:13:50.538472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:50.551169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c5b02-2cf7-451a-bd39-5f65571a6e8c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:50.569118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:50.571518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:50.573408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:50.575272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:50.576117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:50.582216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:50.612685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 891/1745] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:50.614023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:50.627799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e866d-3ee9-44da-a497-b1986df3e810 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:50.630865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:50.634454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:50.643850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:50.645558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:50.646192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:50.647725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:50.649481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] PATH_INFO: `/v3/projects/2bfdd0f233914ec8aa9dabf87d91170b/users/c93397fec3d849f9934743935ad5a4b2/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:50.699109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfdd0f233914ec8aa9dabf87d91170b, user_id=c93397fec3d849f9934743935ad5a4b2, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:50.702982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:50.745576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 855/1746] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:50 2022] PUT /identity/v3/projects/2bfdd0f233914ec8aa9dabf87d91170b/users/c93397fec3d849f9934743935ad5a4b2/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:50.749612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:50.859346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fad9e-d6d5-44c9-8f73-9bbea715f251 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:51.023547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134f3a-e6d2-4eb8-a58c-8fc90c036258 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:51.054959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:51.056162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.056461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.056745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.057487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:51.058650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8afc10-7491-4a7b-b603-d7d6e01b186f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:51.060459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8afc10-7491-4a7b-b603-d7d6e01b186f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.060791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8afc10-7491-4a7b-b603-d7d6e01b186f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.061109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8afc10-7491-4a7b-b603-d7d6e01b186f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.064010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:51.106896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 856/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:51.108172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:51.148655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782b5ab-9c1c-4878-ac93-be8ca0a794ac admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:51.149044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8afc10-7491-4a7b-b603-d7d6e01b186f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:51.158548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8afc10-7491-4a7b-b603-d7d6e01b186f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:51.163708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a348868-b2ef-4179-af7b-3c9da6d1006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:51.166755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a348868-b2ef-4179-af7b-3c9da6d1006a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.176800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a348868-b2ef-4179-af7b-3c9da6d1006a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.176800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a348868-b2ef-4179-af7b-3c9da6d1006a admin admin] PATH_INFO: `/v3/projects/2bfdd0f233914ec8aa9dabf87d91170b/users/c93397fec3d849f9934743935ad5a4b2/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.179776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a348868-b2ef-4179-af7b-3c9da6d1006a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfdd0f233914ec8aa9dabf87d91170b, user_id=c93397fec3d849f9934743935ad5a4b2, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:51.182947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a348868-b2ef-4179-af7b-3c9da6d1006a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:51.224385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 892/1748] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:13:51.250057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.258169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 857/1749] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:51 2022] PUT /identity/v3/projects/2bfdd0f233914ec8aa9dabf87d91170b/users/c93397fec3d849f9934743935ad5a4b2/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:51.265347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.268106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.269653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.270474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.271022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.364558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] MFA Rules not processed for user `c93397fec3d849f9934743935ad5a4b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:51.366102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:51.387222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:51.409005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] MFA Rules not processed for user `018eb80709e94d5e8653fb6eeb342c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:51.414399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:51.639570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 858/1750] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:51.639570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:51.648216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:51.741613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 893/1751] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:51.743003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:51.751838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:51.842006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34acea-2bdb-4a22-b5eb-48caedf7acba None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:51.856699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:51.862341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.862974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.863420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] PATH_INFO: `/v3/projects/e0b875c08c024b30bd641438f5d4e56b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.870000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b875c08c024b30bd641438f5d4e56b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:51.875356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:51.894752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3168f-fb0d-46d1-9a9c-aaa64e87d621 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:51.900021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d439fe2-d71a-4f02-a0ad-59d79aba2212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:51.900834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d439fe2-d71a-4f02-a0ad-59d79aba2212 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.901228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d439fe2-d71a-4f02-a0ad-59d79aba2212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.901654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d439fe2-d71a-4f02-a0ad-59d79aba2212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:51.921815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:51.929669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d439fe2-d71a-4f02-a0ad-59d79aba2212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:51.931352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d439fe2-d71a-4f02-a0ad-59d79aba2212 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:51.944217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 894/1752] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:51.970317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:51.972618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:51.973275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:52.048595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] MFA Rules not processed for user `018eb80709e94d5e8653fb6eeb342c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:52.050355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:52.133458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b875c08c024b30bd641438f5d4e56b', 'request_id': 'req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:52.173357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b875c08c024b30bd641438f5d4e56b', 'request_id': 'req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:52.181939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 895/1753] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:52.181939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:52.190761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b875c08c024b30bd641438f5d4e56b', 'request_id': 'req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:52.236705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] The token cache is being invalidate because project e0b875c08c024b30bd641438f5d4e56b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:52.238120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:52.240812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:52.250635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 859/1754] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:51 2022] DELETE /identity/v3/projects/e0b875c08c024b30bd641438f5d4e56b => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:52.254315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:52.258643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:52.262282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707af14-f314-4d2e-98ac-aca698d0b7b8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:52.266528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:52.267311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:52.267447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:52.268176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:52.275312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:52.392532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:52.542253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:52.542253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:52.542427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6eaec-a0ef-4f28-ac10-7068517c1c7a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:52.587937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 896/1755] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:52.589237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:52.592739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:52.608667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:52.612191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:52.614499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:52.614804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:52.633978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:52.691902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:52.693857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dd791-87bb-4799-a178-c3cf17f16993 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:52.724270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 860/1756] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:52.794378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:52.829885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:52.830177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:52.830517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:52.881374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53743b07-54cc-4c6d-b98a-4e9c3e256fc3 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:52.935057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:52.935711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:52.936005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:52.936372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] PATH_INFO: `/v3/users/416a26ca40234b51acb7763dcf7de372` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:52.937144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] MFA Rules not processed for user `c8ab1df6f15143de8f922a626317fa9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:52.938381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:52.947551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=416a26ca40234b51acb7763dcf7de372)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:52.953120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:53.076551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416a26ca40234b51acb7763dcf7de372'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:53.079831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416a26ca40234b51acb7763dcf7de372'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:53.081262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416a26ca40234b51acb7763dcf7de372'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:53.109963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416a26ca40234b51acb7763dcf7de372'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:53.161619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416a26ca40234b51acb7763dcf7de372'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:53.216482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 897/1757] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:52 2022] DELETE /identity/v3/users/416a26ca40234b51acb7763dcf7de372 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:53.309133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 861/1758] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:53.436196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:53.438761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:53.648127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a730e-42c9-422b-8e30-06f78a0b40ee admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:53.668549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a197df-19ec-41ab-b81b-0e7fa6373005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:53.670494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a197df-19ec-41ab-b81b-0e7fa6373005 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:53.680840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a197df-19ec-41ab-b81b-0e7fa6373005 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:53.689885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a197df-19ec-41ab-b81b-0e7fa6373005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:53.845116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a197df-19ec-41ab-b81b-0e7fa6373005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:53.845901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:53.848383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:53.849271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a197df-19ec-41ab-b81b-0e7fa6373005 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:53.868689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 898/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:13:53.895661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f54eb-237a-4bb6-80e5-0b71e75eb6b5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:54.163348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:54.164066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:54.164355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:54.164636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] PATH_INFO: `/v3/projects/1840ddc234704ee18e9ab9b927ec10bf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:54.170227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1840ddc234704ee18e9ab9b927ec10bf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:54.172260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:54.259471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaee05-6e42-4a9a-84e1-64310543ab44 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:54.260468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaee05-6e42-4a9a-84e1-64310543ab44 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:54.261001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaee05-6e42-4a9a-84e1-64310543ab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:54.314623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:54.352896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bfeaee05-6e42-4a9a-84e1-64310543ab44 None None] MFA Rules not processed for user `166cef282cb24a5aaef0ca8b28f02e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:54.355627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bfeaee05-6e42-4a9a-84e1-64310543ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:54.368390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1840ddc234704ee18e9ab9b927ec10bf', 'request_id': 'req-8c444de2-3f74-41fd-b62b-c1ff02c3008e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.376501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:54.390349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1840ddc234704ee18e9ab9b927ec10bf', 'request_id': 'req-8c444de2-3f74-41fd-b62b-c1ff02c3008e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.414778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1840ddc234704ee18e9ab9b927ec10bf', 'request_id': 'req-8c444de2-3f74-41fd-b62b-c1ff02c3008e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.450370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] The token cache is being invalidate because project 1840ddc234704ee18e9ab9b927ec10bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:54.450370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.450622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.457394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 862/1760] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:53 2022] DELETE /identity/v3/projects/1840ddc234704ee18e9ab9b927ec10bf => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:13:54.458787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:54.460956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:54.538194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 899/1761] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:54.654186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7b3a-9cd4-4215-a116-b7030aac2d76 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:54.654486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7b3a-9cd4-4215-a116-b7030aac2d76 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:54.659623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7b3a-9cd4-4215-a116-b7030aac2d76 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:54.672481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 900/1762] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:13:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:13:54.696482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c444de2-3f74-41fd-b62b-c1ff02c3008e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:54.698109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:54.698819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:54.699158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:54.699440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] PATH_INFO: `/v3/users/abdb989a0bb147758148f43251b71989` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:54.701662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdb989a0bb147758148f43251b71989)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:54.703687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:54.788296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:54.788698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:54.789095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:54.851536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdb989a0bb147758148f43251b71989'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.855843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:54.857280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:54.865089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdb989a0bb147758148f43251b71989'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.866313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdb989a0bb147758148f43251b71989'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.927356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdb989a0bb147758148f43251b71989'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:54.983244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdb989a0bb147758148f43251b71989'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:55.013656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 863/1763] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:54 2022] DELETE /identity/v3/users/abdb989a0bb147758148f43251b71989 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:55.155536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 901/1764] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:13:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:13:56.142993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:56.145511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:56.433439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:56.436012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:56.481676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:56.487646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c4d18-7f6f-4fd0-945d-6df438ce4bdd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:56.513264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f95e0c-cb83-4b49-8b43-4d63d905da7a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:56.535672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:56.543110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:56.546201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:56.547210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] PATH_INFO: `/v3/projects/d517899c0f604866b1e0229dcd42818d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:56.560882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d517899c0f604866b1e0229dcd42818d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:56.563559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:56.771478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:56.772428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:56.773016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:56.773434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:56.778252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:56.800343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd517899c0f604866b1e0229dcd42818d', 'request_id': 'req-d60d6232-b47d-4aae-9783-4f7c252aca3d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:56.834023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd517899c0f604866b1e0229dcd42818d', 'request_id': 'req-d60d6232-b47d-4aae-9783-4f7c252aca3d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:56.867606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd517899c0f604866b1e0229dcd42818d', 'request_id': 'req-d60d6232-b47d-4aae-9783-4f7c252aca3d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:56.901304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:56.918776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] The token cache is being invalidate because project d517899c0f604866b1e0229dcd42818d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:56.923181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:56.925818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:56.939269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 864/1765] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:56 2022] DELETE /identity/v3/projects/d517899c0f604866b1e0229dcd42818d => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:56.940328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:56.968924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:57.079869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:57.093063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9e88a-364a-476f-b833-0e2364919880 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:57.133016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 902/1766] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:57.136268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:57.136268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:57.136584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:57.220988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:57.225840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:57.358593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d6232-b47d-4aae-9783-4f7c252aca3d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:57.392333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:57.392730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:57.393012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:57.393288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] PATH_INFO: `/v3/projects/455c7947fbb94a00b9c24eb8bf305a75` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:57.401569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=455c7947fbb94a00b9c24eb8bf305a75)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:57.411840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:57.626284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455c7947fbb94a00b9c24eb8bf305a75', 'request_id': 'req-39a34931-aec9-4f5e-91be-73abd8256ea1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:57.637105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 903/1767] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:57.637105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:57.637105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:57.666238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455c7947fbb94a00b9c24eb8bf305a75', 'request_id': 'req-39a34931-aec9-4f5e-91be-73abd8256ea1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:57.706098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455c7947fbb94a00b9c24eb8bf305a75', 'request_id': 'req-39a34931-aec9-4f5e-91be-73abd8256ea1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:57.736340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] The token cache is being invalidate because project 455c7947fbb94a00b9c24eb8bf305a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:13:57.737168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:57.739499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-39a34931-aec9-4f5e-91be-73abd8256ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:57.745287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 865/1768] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:13:56 2022] DELETE /identity/v3/projects/455c7947fbb94a00b9c24eb8bf305a75 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:57.747677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858f47-18e9-41c0-8bd9-4a65b6107c64 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:57.748717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858f47-18e9-41c0-8bd9-4a65b6107c64 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:57.749217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858f47-18e9-41c0-8bd9-4a65b6107c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:57.777292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-89858f47-18e9-41c0-8bd9-4a65b6107c64 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:57.779070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-89858f47-18e9-41c0-8bd9-4a65b6107c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:57.795908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:57.875662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 866/1769] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:57.880801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:57.882037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:57.882549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:57.945679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92692dba-561d-4bcc-a068-e86f9a14163e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:57.948990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:57.952734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:57.994413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 867/1770] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:57.995419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:58.016407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c6c98-390b-4fdc-8c45-49291e5237b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:58.031876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:58.034342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.035414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.036756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.038700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:58.041022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:58.045647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f44e05-82a6-4222-b1a9-3a5d620de031 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:58.090662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 868/1771] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:58.094798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.095919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.096207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.164053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:58.166573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:58.167391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:58.194010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:58.194873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.195298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.199256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.199433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 869/1772] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:58.199433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:58.211851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:58.234446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f4a7c-a641-4d36-8b45-75fc176a2a55 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:58.249654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:58.251517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5abe9-bd3b-4099-a28b-e464a212e27f service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:58.253412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:58.258112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.260108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.261116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.264562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:58.271971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:58.294404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 904/1773] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:13:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:13:58.297986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.304525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.305697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.388866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 870/1774] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:58 2022] POST /identity/v3/projects => generated 399 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:58.389759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:58.406754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] MFA Rules not processed for user `018eb80709e94d5e8653fb6eeb342c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:58.416819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:58.696761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 905/1775] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:58.707800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:58.710392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83be3-13e7-42fd-a412-e86212d0e27d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:58.738721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:58.741018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.750122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.750636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.751980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:13:58.755404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:58.765792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b30df-3c18-4354-96f3-e5c52a0bf57e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:58.769666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:58.808941 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:58.833768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.834052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.835415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.836187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:58.846086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:58.866628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 871/1776] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:13:58.871151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:58.923947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a778d-315b-4212-a2f8-c6b539fe5abf admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:58.963468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 906/1777] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:58 2022] POST /identity/v3/users => generated 300 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:58.964491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:58.967623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:58.968407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:58.968865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:58.971094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:58.973634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:58.978149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:59.000400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669a1e3-6e4d-4685-b47f-b8617bc3c90f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:59.016916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:59.017566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.017868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.018141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:59.021207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:59.024546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:59.050009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 907/1778] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:59.057750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:59.057936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 872/1779] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:13:58 2022] POST /identity/v3/projects => generated 391 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:59.060247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:59.089302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ce92b-9294-4526-9273-da8d24e40f55 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:59.190496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:59.271789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf2c9d-9668-45d6-b539-2b3f102c911e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:59.290774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:59.293479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.294490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:59.295469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.296210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.296667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:59.297498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:59.297999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.298427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:59.301485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:59.304765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:59.305177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:59.339414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 908/1780] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:13:59.342161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:59.348823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:59.372989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 873/1781] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:13:59 2022] POST /identity/v3/users => generated 296 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:13:59.372989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:59.392606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900102e7-4e35-4d5d-b503-afa1cba20e7e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:59.401214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91712-bdbc-4fe4-aa9b-3ed81fc50f5b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:59.404773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:59.406408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.407561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.409903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] PATH_INFO: `/v3/users/9fb6e2a043c1414a9fb4733505604595` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:59.415349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb6e2a043c1414a9fb4733505604595)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:59.417013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:13:59.422607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.423041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:59.423530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.424795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] PATH_INFO: `/v3/projects/515346e82b75476690554ce9389d6dd6/users/b957fba4860441dcb0006f0747a2bddb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:59.526362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=515346e82b75476690554ce9389d6dd6, user_id=b957fba4860441dcb0006f0747a2bddb, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:13:59.579502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:13:59.608278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb6e2a043c1414a9fb4733505604595'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:59.609038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb6e2a043c1414a9fb4733505604595'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:59.609757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb6e2a043c1414a9fb4733505604595'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:59.634439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb6e2a043c1414a9fb4733505604595'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:59.640038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:13:59.672967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-284ec35c-3610-4de0-96cb-9a879b3615cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb6e2a043c1414a9fb4733505604595'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:13:59.684862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 874/1782] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:13:59 2022] PUT /identity/v3/projects/515346e82b75476690554ce9389d6dd6/users/b957fba4860441dcb0006f0747a2bddb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:59.688756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:13:59.692238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:59.716342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 909/1783] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:13:59 2022] DELETE /identity/v3/users/9fb6e2a043c1414a9fb4733505604595 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:13:59.718132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09711514-b244-4158-b913-a071ce860497 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.718582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09711514-b244-4158-b913-a071ce860497 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.719148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09711514-b244-4158-b913-a071ce860497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:13:59.775915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-09711514-b244-4158-b913-a071ce860497 None None] MFA Rules not processed for user `b957fba4860441dcb0006f0747a2bddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:13:59.778532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-09711514-b244-4158-b913-a071ce860497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:13:59.925930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6922-b195-4fbf-92c4-9da5bc7e0ff4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:13:59.971836 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 910/1784] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:13:59.975441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb5ae0-d8c6-4056-8fc0-d634ba87bec5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:13:59.975775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb5ae0-d8c6-4056-8fc0-d634ba87bec5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:13:59.976081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb5ae0-d8c6-4056-8fc0-d634ba87bec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.022496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-62bb5ae0-d8c6-4056-8fc0-d634ba87bec5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:00.023809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-62bb5ae0-d8c6-4056-8fc0-d634ba87bec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:00.084892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.085567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.085858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.086141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.086942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.089138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.113802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 875/1785] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:00.117251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:00.124505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:00.150979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 911/1786] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:00.157215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.157848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.158324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.174083 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2535a42-9338-48c9-a50c-eecbbbdab454 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:00.182750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.183470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.183761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.184148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.184857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.186964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.210991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] MFA Rules not processed for user `c8ab1df6f15143de8f922a626317fa9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:00.212641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:00.255166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 876/1787] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:00 2022] POST /identity/v3/projects => generated 401 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:00.261940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:00.321984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 912/1788] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:00.323468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:00.324950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:00.368793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:00.369740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:00.493984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa9c25-eae3-46ca-9b9b-1d2e5cb4bdfb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:00.528135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d8c1-b7cf-47fe-90bf-cbe8b10d8e0a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:00.532189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.536046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.536391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.536682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.539707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.546898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.591807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 877/1789] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:00.593316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:00.608649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66575ac-527e-457c-bf6d-b731e1427c83 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:00.622716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.623790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.624589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.624876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.625633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.628559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.663584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 878/1790] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:00 2022] POST /identity/v3/users => generated 307 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:00.665411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:00.683138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcff31-9380-41d6-9b8a-57029c9cc127 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:00.684623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.685343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.685685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.685997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.689068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b60bc-5d85-4856-9967-5127ac08df8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.689836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b60bc-5d85-4856-9967-5127ac08df8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.690129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b60bc-5d85-4856-9967-5127ac08df8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.690464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b60bc-5d85-4856-9967-5127ac08df8b admin admin] PATH_INFO: `/v3/projects/0335447549e24ae69dd3badd44bf8ce3/users/059c21df50064745a28ad67a272a718f/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.710728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b60bc-5d85-4856-9967-5127ac08df8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0335447549e24ae69dd3badd44bf8ce3, user_id=059c21df50064745a28ad67a272a718f, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.714401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b60bc-5d85-4856-9967-5127ac08df8b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.715691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.720370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.757740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 913/1791] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:00.761136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:00.761873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 879/1792] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:00 2022] PUT /identity/v3/projects/0335447549e24ae69dd3badd44bf8ce3/users/059c21df50064745a28ad67a272a718f/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:00.778458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.778766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.779082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.798971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b51ed1-b635-4a30-a763-b702d684187b service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:00.856729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] MFA Rules not processed for user `059c21df50064745a28ad67a272a718f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:00.864338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:00.980074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:00.981194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:00.981783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:00.982262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:00.983548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:00.985520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:00.998213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 914/1793] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:01.002439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.014811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f306-7a88-4ea6-9c8c-80b9b542f2c4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:01.021505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:01.022859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.023547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.024047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] PATH_INFO: `/v3/projects/4c5daca07abc4765a34457bdbdf77b87/users/50b2f624b1d84673bbc8b239ff85d43f/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.044540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5daca07abc4765a34457bdbdf77b87, user_id=50b2f624b1d84673bbc8b239ff85d43f, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:01.048005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:01.082255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 915/1794] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:01 2022] PUT /identity/v3/projects/4c5daca07abc4765a34457bdbdf77b87/users/50b2f624b1d84673bbc8b239ff85d43f/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:01.083869 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 880/1795] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:01.085235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.086620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.097468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a14a8-7778-4ce5-9a72-b25e9953e575 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:01.211275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:01.265879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd92fd-4be5-4e61-bcac-45b5978b25ee admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:01.274657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:01.275627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.276061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.276500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.276631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:01.277428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.277634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:01.279772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:01.281053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.281488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.282356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:01.290468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:01.334676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 916/1796] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:01.340439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:01.344812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.352383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 881/1797] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:01 2022] POST /identity/v3/projects => generated 393 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:01.360357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.370968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0af206-7d81-46a9-b89a-5044ce1a145a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:01.569130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3c14-a873-442c-9dc1-d64913ad8c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:01.570392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3c14-a873-442c-9dc1-d64913ad8c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.570392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3c14-a873-442c-9dc1-d64913ad8c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.570827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3c14-a873-442c-9dc1-d64913ad8c33 admin admin] PATH_INFO: `/v3/projects/4c5daca07abc4765a34457bdbdf77b87/users/50b2f624b1d84673bbc8b239ff85d43f/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.571881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bd882-dd65-4ca1-962c-b9aa57204a6d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:01.573362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3c14-a873-442c-9dc1-d64913ad8c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5daca07abc4765a34457bdbdf77b87, user_id=50b2f624b1d84673bbc8b239ff85d43f, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:01.576584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3c14-a873-442c-9dc1-d64913ad8c33 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:01.580271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b711-8326-4a87-9857-c4b3344ca36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:01.581585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b711-8326-4a87-9857-c4b3344ca36e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.582103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b711-8326-4a87-9857-c4b3344ca36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.584131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b711-8326-4a87-9857-c4b3344ca36e admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.585187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b711-8326-4a87-9857-c4b3344ca36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:01.594867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b711-8326-4a87-9857-c4b3344ca36e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:01.615155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 917/1798] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:01 2022] PUT /identity/v3/projects/4c5daca07abc4765a34457bdbdf77b87/users/50b2f624b1d84673bbc8b239ff85d43f/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:01.617150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d49a9-6fe1-4b41-ab7e-5349ed5b37ef None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.617597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d49a9-6fe1-4b41-ab7e-5349ed5b37ef None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.617895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d49a9-6fe1-4b41-ab7e-5349ed5b37ef None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.622429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 918/1799] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jul 28 01:14:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:14:01.627076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.627686 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.627980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.665069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 882/1800] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:01 2022] POST /identity/v3/users => generated 303 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:01.667627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:01.668629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:01.669164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:01.716421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] MFA Rules not processed for user `50b2f624b1d84673bbc8b239ff85d43f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:01.727498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:01.742995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:01.749792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:01.903206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 919/1801] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:01.904222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.929414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 883/1802] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:14:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:01.930644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:01.932371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.051445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:02.059028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ee15-22ec-4ef0-bb98-531594647aea None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.065895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.067235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.067631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.068110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.069119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.071036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.100425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 920/1803] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:02.101542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.105165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.118679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ada45-10af-42db-8b4e-d11a576d7ed6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.132454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.133506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.136060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.136573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] PATH_INFO: `/v3/projects/6336fa83cba644809833f46e4f259be3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.139217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6336fa83cba644809833f46e4f259be3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.142740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.220386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:02.237703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6336fa83cba644809833f46e4f259be3', 'request_id': 'req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.262786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6336fa83cba644809833f46e4f259be3', 'request_id': 'req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.270539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504855ce-16fa-4ee5-845d-3be8c4fafffe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.271265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6336fa83cba644809833f46e4f259be3', 'request_id': 'req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.301084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] The token cache is being invalidate because project 6336fa83cba644809833f46e4f259be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:02.301366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.302279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.304609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 884/1804] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:01 2022] DELETE /identity/v3/projects/6336fa83cba644809833f46e4f259be3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:02.306664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.308299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.395629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.401328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.401625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.402020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.406006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.442987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824b3ff-d339-4d9f-a2c6-d5046c944ae3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.456328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.457423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.458195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.458195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.460588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.475234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.476765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.490223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 885/1805] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:02.491157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.496915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.561144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.562779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.597513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0972b-e847-43bb-b219-daa5106f3f49 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.599265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 921/1806] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:02.600348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.604233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.605770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.606995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.607483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.607960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] PATH_INFO: `/v3/projects/a29411cc03f54edd9273ec6a46fa4834/users/e44f0d82a7674b3b8b0004ec7fddef48/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.643096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29411cc03f54edd9273ec6a46fa4834, user_id=e44f0d82a7674b3b8b0004ec7fddef48, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.668398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.669681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d5b30-3882-499c-8a21-ab856abe6207 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.687976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.688679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.688968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.689251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] PATH_INFO: `/v3/users/24f684c7588e4c10933ba799a8cdc995` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.690989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 886/1807] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:02 2022] PUT /identity/v3/projects/a29411cc03f54edd9273ec6a46fa4834/users/e44f0d82a7674b3b8b0004ec7fddef48/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:02.693989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.694703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f684c7588e4c10933ba799a8cdc995)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.700771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.810578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:02.843601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beaffb-f2d1-48de-ba99-e0ad82c7d76d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:02.858047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:02.858712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:02.859668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:02.859968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:02.860719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:02.863974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f684c7588e4c10933ba799a8cdc995'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.864692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f684c7588e4c10933ba799a8cdc995'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.865785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f684c7588e4c10933ba799a8cdc995'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.866597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:02.883537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:02.894903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 887/1808] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:02.895128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f684c7588e4c10933ba799a8cdc995'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.895925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.898759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:02.942543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f684c7588e4c10933ba799a8cdc995'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:02.987595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 922/1809] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:02 2022] DELETE /identity/v3/users/24f684c7588e4c10933ba799a8cdc995 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:02.988514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:02.998215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:03.144711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f4234-6660-4dac-aac0-8a642c25cf9b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:03.197942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f60c2-d6cb-4287-8a19-4cb7eaa32783 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:03.212236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2e469-bef9-4302-8687-e40d7eaa73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:03.212981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e469-bef9-4302-8687-e40d7eaa73d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:03.213276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e469-bef9-4302-8687-e40d7eaa73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:03.213541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e469-bef9-4302-8687-e40d7eaa73d3 admin admin] PATH_INFO: `/v3/projects/a29411cc03f54edd9273ec6a46fa4834/users/e44f0d82a7674b3b8b0004ec7fddef48/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:03.218558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2e469-bef9-4302-8687-e40d7eaa73d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29411cc03f54edd9273ec6a46fa4834, user_id=e44f0d82a7674b3b8b0004ec7fddef48, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:03.228569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2e469-bef9-4302-8687-e40d7eaa73d3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:03.264334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:03.268243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:03.268546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:03.269053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:03.272600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:03.303097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 923/1810] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:02 2022] PUT /identity/v3/projects/a29411cc03f54edd9273ec6a46fa4834/users/e44f0d82a7674b3b8b0004ec7fddef48/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:03.315510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:03.315931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:03.316343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:03.404642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:03.406417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:03.438299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] MFA Rules not processed for user `e44f0d82a7674b3b8b0004ec7fddef48`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:03.438704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 888/1811] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:03.446218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:03.728521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 924/1812] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:03.741596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:03.743421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:03.761190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859976e1-0d08-4aab-b54c-6f89e8146289 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:03.901773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1168e-b5d1-4a4f-adb2-c7a69eaa9ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:03.902486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1168e-b5d1-4a4f-adb2-c7a69eaa9ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:03.902781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1168e-b5d1-4a4f-adb2-c7a69eaa9ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:03.905552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1168e-b5d1-4a4f-adb2-c7a69eaa9ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:03.936402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1168e-b5d1-4a4f-adb2-c7a69eaa9ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:03.937913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1168e-b5d1-4a4f-adb2-c7a69eaa9ba2 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:03.981058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 889/1813] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:04.106147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:04.132199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b53cb8-bce0-4b69-bcca-0a8b1c1e670b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:04.150393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:04.151088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:04.151426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:04.151715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:04.158555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:04.352148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:04.388816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:04.389912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:04.413919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 925/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:05.972682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf5b67-bfbf-4863-96f5-a0f6d4825efa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:05.973544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf5b67-bfbf-4863-96f5-a0f6d4825efa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:05.973968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf5b67-bfbf-4863-96f5-a0f6d4825efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:05.981937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:06.014325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-00cf5b67-bfbf-4863-96f5-a0f6d4825efa None None] MFA Rules not processed for user `166cef282cb24a5aaef0ca8b28f02e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:06.022258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-00cf5b67-bfbf-4863-96f5-a0f6d4825efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:06.112952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:06.114906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:06.207569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 890/1815] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:06.291605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512715-6f8d-4a7f-87fd-271d7ba49387 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:06.302059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:06.302952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:06.303300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:06.303583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] PATH_INFO: `/v3/projects/e690da75cdd7412cb975c4a9799d9fce` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:06.305412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e690da75cdd7412cb975c4a9799d9fce)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:06.309209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:06.413025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e690da75cdd7412cb975c4a9799d9fce', 'request_id': 'req-2a7906c0-c67b-44ba-bb4b-029fd2325997'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.427258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e690da75cdd7412cb975c4a9799d9fce', 'request_id': 'req-2a7906c0-c67b-44ba-bb4b-029fd2325997'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.449889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e690da75cdd7412cb975c4a9799d9fce', 'request_id': 'req-2a7906c0-c67b-44ba-bb4b-029fd2325997'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.478399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] The token cache is being invalidate because project e690da75cdd7412cb975c4a9799d9fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:06.478706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.483312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.485708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 926/1816] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:06 2022] DELETE /identity/v3/projects/e690da75cdd7412cb975c4a9799d9fce => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:06.491008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e31b36-25b4-4726-8415-1f537469af24 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:06.491362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e31b36-25b4-4726-8415-1f537469af24 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:06.491647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e31b36-25b4-4726-8415-1f537469af24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:06.546256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-86e31b36-25b4-4726-8415-1f537469af24 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:06.547539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-86e31b36-25b4-4726-8415-1f537469af24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:06.712859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:06.714731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:06.742023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 891/1817] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:06.744108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:06.744403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:06.744689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:06.753123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7906c0-c67b-44ba-bb4b-029fd2325997 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:06.768870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:06.769567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:06.769858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:06.770135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] PATH_INFO: `/v3/users/ef46d65717de42e49871e4431f6fa028` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:06.784186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef46d65717de42e49871e4431f6fa028)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:06.786657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:06.816594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:06.819215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:06.855439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 892/1818] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:06.856654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:06.870594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce9926-4e59-41ec-bc66-db4f96389e3a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:06.889892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:06.890482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:06.890771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:06.891103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:06.891845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:06.892156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:06.893816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:06.928473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef46d65717de42e49871e4431f6fa028'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.929275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef46d65717de42e49871e4431f6fa028'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.929897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef46d65717de42e49871e4431f6fa028'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.936900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 893/1819] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:14:06.937916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:06.942179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef46d65717de42e49871e4431f6fa028'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.951788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:06.960096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011a4636-53f2-425d-9ec2-f7edb06ee9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef46d65717de42e49871e4431f6fa028'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:06.963243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:06.983047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd3a38-67a0-433f-a910-f5169fb5c94a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:07.001392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:07.002401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.002876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.003340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.009893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 927/1820] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:06 2022] DELETE /identity/v3/users/ef46d65717de42e49871e4431f6fa028 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:07.010671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:07.013922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:07.016030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.016366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.016661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.078029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:07.081084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:07.135550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 894/1821] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:06 2022] POST /identity/v3/projects => generated 357 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:07.141025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:07.347304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 928/1822] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:07.402797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b36ee-f7bb-4b9f-9847-cc8f43abbf32 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:07.419021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:07.419957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.420322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.420783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.421583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:07.423679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:07.476587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:07.503076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 895/1823] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:07 2022] POST /identity/v3/users => generated 279 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:07.503535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:07.516524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673045cb-d41f-4f09-b914-f3754e901a40 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:07.536192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:07.536850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.537132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.537415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.543660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:07.557115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:07.611448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 929/1824] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:07.615921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:07.642159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896068b7-c088-402c-a6b5-678659bdfdd9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:07.658573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:07.659270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.659561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.659843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.665239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:07.666958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:07.708791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:07.715825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 896/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:07.721950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a426b4f-6966-43da-aae0-1fb3c7c397c5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:07.737395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d9376-66c8-4b1f-999b-fae554b0201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:07.737802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d9376-66c8-4b1f-999b-fae554b0201e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.738088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d9376-66c8-4b1f-999b-fae554b0201e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.738366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d9376-66c8-4b1f-999b-fae554b0201e admin admin] PATH_INFO: `/v3/projects/3680707d566b43ad924b84e80b851b64/users/a665f3e557e442ed990d56161749f0c1/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.799188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d9376-66c8-4b1f-999b-fae554b0201e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3680707d566b43ad924b84e80b851b64, user_id=a665f3e557e442ed990d56161749f0c1, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:07.805849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d9376-66c8-4b1f-999b-fae554b0201e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:07.808710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:07.816359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:07.854724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 930/1826] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:07 2022] PUT /identity/v3/projects/3680707d566b43ad924b84e80b851b64/users/a665f3e557e442ed990d56161749f0c1/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:07.865204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:07.865440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:07.865807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:07.961744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] MFA Rules not processed for user `a665f3e557e442ed990d56161749f0c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:07.963209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:07.973313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a6166-d618-4183-b5da-e51884f0dcf6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:08.132109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:08.132810 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.133136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.133453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.210462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 931/1827] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:08.212046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:08.298279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:08.299537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:08.325578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:08.334448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 897/1828] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:08.367495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854e7cc-0a2e-43ba-a029-90d0c04616a3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:08.374783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983287a4-1ca5-4d63-9839-9f2cc249808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:08.375761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983287a4-1ca5-4d63-9839-9f2cc249808b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.376112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983287a4-1ca5-4d63-9839-9f2cc249808b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.376460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983287a4-1ca5-4d63-9839-9f2cc249808b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.377337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983287a4-1ca5-4d63-9839-9f2cc249808b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:08.380245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983287a4-1ca5-4d63-9839-9f2cc249808b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:08.497621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 932/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:08 2022] POST /identity/v3/projects => generated 355 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:08.498873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:08.513931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cea282-12e2-4242-a0f1-03dd612e0b1f service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:08.622444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed9418-3d14-47db-914a-449502ea86c8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.623145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed9418-3d14-47db-914a-449502ea86c8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.623586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed9418-3d14-47db-914a-449502ea86c8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.626676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 933/1830] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:14:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 01:14:08.645748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:08.667443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.667731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.668019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.681343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:08.683098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.683512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.683724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.684770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:08.687823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:08.730934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 898/1831] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:08 2022] POST /identity/v3/users => generated 284 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:08.731626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:08.733007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:08.733386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:08.761711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318502fa-610b-4924-8263-36c97f4ab30b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:08.778581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:08.779471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.782024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.791893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.793898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:08.804466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:08.844153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 899/1832] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:08.845624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:08.862138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47db1-5639-4a89-ad2d-e088b677df72 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:08.899625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:08.900409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:08.900697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:08.901002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] PATH_INFO: `/v3/projects/1cd526b84def4794b7268cb5575f05ab/users/a887d9d4f38c4dd88c945e08331c3f15/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:08.949062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd526b84def4794b7268cb5575f05ab, user_id=a887d9d4f38c4dd88c945e08331c3f15, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:08.951698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:08.968527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 934/1833] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:14:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:08.985665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 900/1834] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:08 2022] PUT /identity/v3/projects/1cd526b84def4794b7268cb5575f05ab/users/a887d9d4f38c4dd88c945e08331c3f15/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:08.991242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:09.005475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fcd4-25e9-41ca-b032-cc8b31f325ed None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:09.114890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:09.115808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:09.116224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:09.116519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:09.118080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:09.120600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:09.136721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 935/1835] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:09.140717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:09.171711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1710a7-f962-4200-8613-67ca66db3e66 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:09.186610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e64d1-011e-4cb7-8bf8-747358a47804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:09.187356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e64d1-011e-4cb7-8bf8-747358a47804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:09.187639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e64d1-011e-4cb7-8bf8-747358a47804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:09.187998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e64d1-011e-4cb7-8bf8-747358a47804 admin admin] PATH_INFO: `/v3/projects/1cd526b84def4794b7268cb5575f05ab/users/a887d9d4f38c4dd88c945e08331c3f15/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:09.191818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e64d1-011e-4cb7-8bf8-747358a47804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd526b84def4794b7268cb5575f05ab, user_id=a887d9d4f38c4dd88c945e08331c3f15, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:09.195269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e64d1-011e-4cb7-8bf8-747358a47804 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:09.226997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:09.236442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:09.239418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 901/1836] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:09 2022] PUT /identity/v3/projects/1cd526b84def4794b7268cb5575f05ab/users/a887d9d4f38c4dd88c945e08331c3f15/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:09.258236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:09.258733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:09.259178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:09.332010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:09.335797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] MFA Rules not processed for user `a887d9d4f38c4dd88c945e08331c3f15`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:09.337257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:09.363623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331a49d-e02e-4cf3-a0df-c650c9d76370 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:09.476952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 902/1837] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:09 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:09.477897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:09.481584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:09.504430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:09.505498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:09.505753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:09.506030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:09.528729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:09.531068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:09.548339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 936/1838] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:09.549588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:09.553662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:09.638987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b1cb-0f8c-405a-b3b7-b7b09d497d28 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:09.651511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:09.652421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:09.654253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:09.654924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] PATH_INFO: `/v3/users/b957fba4860441dcb0006f0747a2bddb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:09.664061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b957fba4860441dcb0006f0747a2bddb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:09.670288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:09.757969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa690da-ba93-4f35-afda-9d0b65f15085 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:09.780287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:09.781052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:09.784040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:09.786367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:09.814874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:09.816487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:09.843868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 937/1839] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:09 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:09.858389 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:09.919016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b957fba4860441dcb0006f0747a2bddb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:09.938407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b957fba4860441dcb0006f0747a2bddb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:09.941478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b957fba4860441dcb0006f0747a2bddb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:09.988361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b957fba4860441dcb0006f0747a2bddb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.017767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe2fe87b-7f06-4778-9316-366506ccdf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b957fba4860441dcb0006f0747a2bddb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.043864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 903/1840] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:09 2022] DELETE /identity/v3/users/b957fba4860441dcb0006f0747a2bddb => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:10.045776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:10.048418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:10.049000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:10.049851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:10.060782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19f25-493e-4385-89d0-2604cc73bd44 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:10.101063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:10.103283 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:10.191611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:10.192653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:10.193112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:10.193822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:10.196987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:10.204758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 904/1841] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:10.205873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:10.208282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:10.226235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5a3fb-6654-43bb-9b65-af3d3a5eeb7e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:10.231487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:10.232101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:10.232392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:10.233340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] PATH_INFO: `/v3/users/059c21df50064745a28ad67a272a718f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:10.234345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059c21df50064745a28ad67a272a718f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:10.238322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:10.303082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:10.339803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:10.341816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:10.366857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059c21df50064745a28ad67a272a718f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.378363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 938/1842] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:10 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:10.380252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:10.380827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059c21df50064745a28ad67a272a718f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.382641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059c21df50064745a28ad67a272a718f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.386308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:10.417228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059c21df50064745a28ad67a272a718f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.429012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a4951-fb3e-4f8d-97f2-e82c3072e5e2 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:10.429221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059c21df50064745a28ad67a272a718f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.443927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 905/1843] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:10 2022] DELETE /identity/v3/users/059c21df50064745a28ad67a272a718f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:10.448255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:10.448255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:10.448483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:10.449260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:10.450866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:10.450866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] PATH_INFO: `/v3/projects/6b0a23a62635438ba00b3249ca7e7894` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:10.456783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0a23a62635438ba00b3249ca7e7894)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:10.464757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:10.480991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:10.611644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303c5dc-d81a-429a-852b-1a8a2ce37e36 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:10.663702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0a23a62635438ba00b3249ca7e7894', 'request_id': 'req-b04dd64d-5ea2-4806-90ec-4eed191cba4b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.674983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0a23a62635438ba00b3249ca7e7894', 'request_id': 'req-b04dd64d-5ea2-4806-90ec-4eed191cba4b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.725173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0a23a62635438ba00b3249ca7e7894', 'request_id': 'req-b04dd64d-5ea2-4806-90ec-4eed191cba4b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.794809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] The token cache is being invalidate because project 6b0a23a62635438ba00b3249ca7e7894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:10.799947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:10.800623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:10.800983 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:10.801179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:10.804367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.810016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b04dd64d-5ea2-4806-90ec-4eed191cba4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:10.811927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:10.818945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 939/1844] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:10 2022] DELETE /identity/v3/projects/6b0a23a62635438ba00b3249ca7e7894 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:10.831960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252fce3-e45f-491f-80b6-e20792a2a30a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:10.831960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252fce3-e45f-491f-80b6-e20792a2a30a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:10.832331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252fce3-e45f-491f-80b6-e20792a2a30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:10.897797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6252fce3-e45f-491f-80b6-e20792a2a30a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:10.899129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6252fce3-e45f-491f-80b6-e20792a2a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:11.008635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:11.012535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:11.038719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 906/1845] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:11.039907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:11.042157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:11.064984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 940/1846] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:11.066215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2409b13-d3c9-41e3-b27b-ad9d00779e2e service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:11.066664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8237f7-a359-4ce0-8999-015f6e952020 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:11.067050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8237f7-a359-4ce0-8999-015f6e952020 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:11.067337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8237f7-a359-4ce0-8999-015f6e952020 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:11.069840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 941/1847] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:14:11.077174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:11.078483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:11.081224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:11.081642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] PATH_INFO: `/v3/users/c93397fec3d849f9934743935ad5a4b2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:11.096805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93397fec3d849f9934743935ad5a4b2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:11.106403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:11.113569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8237f7-a359-4ce0-8999-015f6e952020 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:11.115932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cf8237f7-a359-4ce0-8999-015f6e952020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:11.147373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8237f7-a359-4ce0-8999-015f6e952020 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:11.158716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83813429-a090-4cef-93f4-39a0c5e260bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:11.159410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83813429-a090-4cef-93f4-39a0c5e260bb service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:11.159697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83813429-a090-4cef-93f4-39a0c5e260bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:11.159977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83813429-a090-4cef-93f4-39a0c5e260bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:11.184952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83813429-a090-4cef-93f4-39a0c5e260bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:11.191293 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83813429-a090-4cef-93f4-39a0c5e260bb service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:11.206087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 942/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:11.207917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:11.208240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:11.208474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:11.229662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93397fec3d849f9934743935ad5a4b2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:11.234505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:11.236362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:11.253350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93397fec3d849f9934743935ad5a4b2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:11.254475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93397fec3d849f9934743935ad5a4b2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:11.273177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:11.275842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93397fec3d849f9934743935ad5a4b2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:11.294001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93397fec3d849f9934743935ad5a4b2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:11.306238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 907/1849] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:11 2022] DELETE /identity/v3/users/c93397fec3d849f9934743935ad5a4b2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:11.351823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:11.387507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 943/1850] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:14:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:11.423535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:11.425663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:11.599833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142afd52-3e43-49ae-af6f-c4e7c4a4d962 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:11.613424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:11.614793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:11.615299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:11.615643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:11.629293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:11.631941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:11.645726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 908/1851] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:12.815960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:12.818906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:12.827294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:12.830592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:12.848499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a4770-6106-4039-b4c2-5b4fff026073 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:12.931278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:12.932420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:12.932912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:12.933393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] PATH_INFO: `/v3/projects/2bfdd0f233914ec8aa9dabf87d91170b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:12.937360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bfdd0f233914ec8aa9dabf87d91170b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:12.944274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:12.979981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76316e4-1e2b-4577-8bc9-e621097eba65 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:12.986459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:12.987327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:12.987718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:12.988100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] PATH_INFO: `/v3/projects/515346e82b75476690554ce9389d6dd6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.003554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] RBAC: Authorizing `identity:delete_project(project_id=515346e82b75476690554ce9389d6dd6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:13.008522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:13.084026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bfdd0f233914ec8aa9dabf87d91170b', 'request_id': 'req-02e2bfb2-c5a0-4840-8a67-08d1194047f1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.108956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bfdd0f233914ec8aa9dabf87d91170b', 'request_id': 'req-02e2bfb2-c5a0-4840-8a67-08d1194047f1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.111641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515346e82b75476690554ce9389d6dd6', 'request_id': 'req-55068f1c-5da3-4e09-b20c-e43a54d2599e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.121991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bfdd0f233914ec8aa9dabf87d91170b', 'request_id': 'req-02e2bfb2-c5a0-4840-8a67-08d1194047f1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.134381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515346e82b75476690554ce9389d6dd6', 'request_id': 'req-55068f1c-5da3-4e09-b20c-e43a54d2599e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.146155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515346e82b75476690554ce9389d6dd6', 'request_id': 'req-55068f1c-5da3-4e09-b20c-e43a54d2599e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.146587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] The token cache is being invalidate because project 2bfdd0f233914ec8aa9dabf87d91170b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:13.146943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.147975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.149996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 944/1852] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:12 2022] DELETE /identity/v3/projects/2bfdd0f233914ec8aa9dabf87d91170b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:13.150861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:13.152374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:13.182903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] The token cache is being invalidate because project 515346e82b75476690554ce9389d6dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:13.185612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.186615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-55068f1c-5da3-4e09-b20c-e43a54d2599e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.202863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 909/1853] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:12 2022] DELETE /identity/v3/projects/515346e82b75476690554ce9389d6dd6 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:13.206114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:13.206614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:13.209243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.219259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:13.241471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:13.245214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:13.267497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2bfb2-c5a0-4840-8a67-08d1194047f1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:13.305818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:13.382941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 910/1854] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:13.383964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:13.386348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:13.400146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:13.400856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef8bf-e0af-4f5d-a2cd-7cdbc8cba877 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:13.401631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:13.402103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:13.402563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] PATH_INFO: `/v3/projects/0335447549e24ae69dd3badd44bf8ce3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.404835 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0335447549e24ae69dd3badd44bf8ce3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:13.405739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:13.406423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:13.406715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:13.407033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] PATH_INFO: `/v3/users/50b2f624b1d84673bbc8b239ff85d43f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.408076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:13.408394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b2f624b1d84673bbc8b239ff85d43f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:13.410337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:13.535094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0335447549e24ae69dd3badd44bf8ce3', 'request_id': 'req-9ac82e56-7a32-4bec-ba72-905f32d56012'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.543844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b2f624b1d84673bbc8b239ff85d43f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.545770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0335447549e24ae69dd3badd44bf8ce3', 'request_id': 'req-9ac82e56-7a32-4bec-ba72-905f32d56012'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.563083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0335447549e24ae69dd3badd44bf8ce3', 'request_id': 'req-9ac82e56-7a32-4bec-ba72-905f32d56012'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.568989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b2f624b1d84673bbc8b239ff85d43f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.569664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b2f624b1d84673bbc8b239ff85d43f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.583457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] The token cache is being invalidate because project 0335447549e24ae69dd3badd44bf8ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:13.583825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.584559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ac82e56-7a32-4bec-ba72-905f32d56012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.587320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 945/1855] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:13 2022] DELETE /identity/v3/projects/0335447549e24ae69dd3badd44bf8ce3 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:13.589025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:13.589329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:13.589614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.599200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b2f624b1d84673bbc8b239ff85d43f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.620305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:13.622000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:13.637104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b2f624b1d84673bbc8b239ff85d43f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:13.656701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 911/1856] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:13 2022] DELETE /identity/v3/users/50b2f624b1d84673bbc8b239ff85d43f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:13.660861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:13.663957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:13.756304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 946/1857] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:13.758065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:13.786953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dca1c3-db07-40b0-b3c7-dd194650efe4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:13.909217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47b292-88fc-4d5c-9983-4f1089042cba admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:13.916248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:13.927135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:13.930709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:13.931863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:13.932153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] PATH_INFO: `/v3/users/e44f0d82a7674b3b8b0004ec7fddef48` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.936424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44f0d82a7674b3b8b0004ec7fddef48)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:13.938687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:13.941207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:13.941913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:13.942199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:13.942610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:13.943531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:13.943868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:13.945729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:13.979362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 947/1858] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:13.984922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:13.989521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:14.001362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0dde3-3a77-4e2e-a03d-b23deec4dbec admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.040270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.042125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.042890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.043525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.044938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.064165 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44f0d82a7674b3b8b0004ec7fddef48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.066629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.086498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44f0d82a7674b3b8b0004ec7fddef48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.088121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44f0d82a7674b3b8b0004ec7fddef48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.108891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44f0d82a7674b3b8b0004ec7fddef48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.133286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 948/1859] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:13 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:14.138252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44f0d82a7674b3b8b0004ec7fddef48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.143062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.151146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:14.159231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 912/1860] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:13 2022] DELETE /identity/v3/users/e44f0d82a7674b3b8b0004ec7fddef48 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:14.261469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232e7e-7507-45f0-89a2-5762bf41d77b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.403201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.404588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.405216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.405702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.408039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.410673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.442921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 949/1861] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:14 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:14.447078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.458336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8d75-eca7-4f01-bf5f-e1e24061e5de admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.463113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.463766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.464058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.464476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.465198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.467442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.488205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 913/1862] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:14.489180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.501721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9909921-d5dd-4d43-9f2b-3b4091408879 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.508089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360768e5-2f84-4041-a74d-46b94c9df333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.508729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360768e5-2f84-4041-a74d-46b94c9df333 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.509015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360768e5-2f84-4041-a74d-46b94c9df333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.509273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360768e5-2f84-4041-a74d-46b94c9df333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.510300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360768e5-2f84-4041-a74d-46b94c9df333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.523973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360768e5-2f84-4041-a74d-46b94c9df333 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.542721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 950/1863] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:14.544088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.555709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1138af-18e1-46a3-a03f-128a3c2ca240 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.562434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.563050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.563335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.563734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] PATH_INFO: `/v3/projects/d93e7712e88744d1a5d6bbe9b305ce66/users/b47d3668a8ca4adfb484c423fc08e0e5/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.584353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93e7712e88744d1a5d6bbe9b305ce66, user_id=b47d3668a8ca4adfb484c423fc08e0e5, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.587531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.617566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 914/1864] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:14 2022] PUT /identity/v3/projects/d93e7712e88744d1a5d6bbe9b305ce66/users/b47d3668a8ca4adfb484c423fc08e0e5/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:14.621048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.621817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.622099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.665480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] MFA Rules not processed for user `b47d3668a8ca4adfb484c423fc08e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:14.666871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:14.677509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.680550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:14.722034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:14.782625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 951/1865] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:14.786713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.799229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f67baf-e717-43da-9303-937158ee7b03 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.805932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.806766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.807087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.807501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] PATH_INFO: `/v3/projects/4c5daca07abc4765a34457bdbdf77b87` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.809378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5daca07abc4765a34457bdbdf77b87)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.811724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.816252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac49c-d280-4ea0-a533-de821c63ddc8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:14.822153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:14.822812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.823319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.823730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:14.824553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:14.826680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:14.879083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 952/1866] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:14 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:14.883927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:14.924271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5daca07abc4765a34457bdbdf77b87', 'request_id': 'req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.945483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5daca07abc4765a34457bdbdf77b87', 'request_id': 'req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.950250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:14.952560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5daca07abc4765a34457bdbdf77b87', 'request_id': 'req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.979917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] The token cache is being invalidate because project 4c5daca07abc4765a34457bdbdf77b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:14.980367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.980930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-010d93c5-87c0-4b5f-bdb4-426c6d2f9eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:14.982713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 915/1867] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:14 2022] DELETE /identity/v3/projects/4c5daca07abc4765a34457bdbdf77b87 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:14.987916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:14.988209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:14.988488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.011296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfba94-a7d7-440e-bbc6-e497635866ae admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.025911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:15.027212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.155301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acf0ad-eaca-4870-9208-52183dc5756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.157483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 916/1868] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:15.158201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.159641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acf0ad-eaca-4870-9208-52183dc5756d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.160699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acf0ad-eaca-4870-9208-52183dc5756d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.161157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acf0ad-eaca-4870-9208-52183dc5756d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.170786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acf0ad-eaca-4870-9208-52183dc5756d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.172537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.181751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acf0ad-eaca-4870-9208-52183dc5756d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.213555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d695b-3aa8-4e2f-aab1-47655c0027d4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.227558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.228242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.228527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.228817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] PATH_INFO: `/v3/projects/a29411cc03f54edd9273ec6a46fa4834` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.232738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 953/1869] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:14 2022] POST /identity/v3/users => generated 289 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:15.234614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.234814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.235227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.237313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29411cc03f54edd9273ec6a46fa4834)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.249235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.326895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:15.331142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.350762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 954/1870] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:15.353304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.357273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.375193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2929a-1f89-4015-86a0-8bef2f54d1bf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.412090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29411cc03f54edd9273ec6a46fa4834', 'request_id': 'req-2362fc90-f817-4d5e-b8f7-38b87625c3d3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:15.438166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29411cc03f54edd9273ec6a46fa4834', 'request_id': 'req-2362fc90-f817-4d5e-b8f7-38b87625c3d3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:15.465755 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29411cc03f54edd9273ec6a46fa4834', 'request_id': 'req-2362fc90-f817-4d5e-b8f7-38b87625c3d3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:15.490508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:15.491974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] The token cache is being invalidate because project a29411cc03f54edd9273ec6a46fa4834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:15.492976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:15.495883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:15.498177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 917/1871] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:15 2022] DELETE /identity/v3/projects/a29411cc03f54edd9273ec6a46fa4834 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:15.500655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.503811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.521557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a7462-ace4-492f-83ae-386fc5fa48e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.523332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a7462-ace4-492f-83ae-386fc5fa48e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.523630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a7462-ace4-492f-83ae-386fc5fa48e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.523912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a7462-ace4-492f-83ae-386fc5fa48e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.529239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a7462-ace4-492f-83ae-386fc5fa48e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.532430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a7462-ace4-492f-83ae-386fc5fa48e7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.569687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 955/1872] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:15.574239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.575477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.577305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.635129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:15.635129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.655472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 956/1873] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:15.657027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.659297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.685324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362fc90-f817-4d5e-b8f7-38b87625c3d3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.689781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be1721-233f-4b67-8341-7e5f1bb5a48d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.700770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.703802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.704079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.704206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.704400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.704802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.705091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.705197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:15.705375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] PATH_INFO: `/v3/projects/bacc28906e91432fb9902c57cc0b6e55/users/4a5df5d9b5ec44ef955a1f08718f96df/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.705477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.707877 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29096-4f4c-4d5c-b6c4-ad5bd64f4970 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.722185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 918/1874] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:15.723551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.723831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.724106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.741264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:15.749089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:15.752652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:15.759259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacc28906e91432fb9902c57cc0b6e55, user_id=4a5df5d9b5ec44ef955a1f08718f96df, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.763250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.767599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 919/1875] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:15.767599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.776306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1618e64-d510-452e-aa02-493799485bf2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.782148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.782936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.783226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.783505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.784288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.786903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.796565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 957/1876] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:15 2022] PUT /identity/v3/projects/bacc28906e91432fb9902c57cc0b6e55/users/4a5df5d9b5ec44ef955a1f08718f96df/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:15.799690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.838615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 920/1877] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:15 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:15.839564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:15.931521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450c775-d05e-43b5-a948-953d388ab465 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:15.957782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:15.958864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:15.959529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:15.959976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:15.961396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:15.963868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:15.983416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 921/1878] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:15.984460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.019996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc410036-8f26-48d9-b403-8bffe318004f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.025197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10453f8f-193f-4b3e-b97f-b40a69aedb88 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.031673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.033609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.035655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.036229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.038619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.041000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.043068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.043849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.043849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.045563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:16.046490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.046986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.049861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.099812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 922/1879] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:15 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:16.101037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.112222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 958/1880] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:16.113657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.142195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047d86-bea0-4a32-b6d5-3393d45486c9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.160794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.162554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.162861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.163233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef85c8-59b8-49a6-a9a2-7a0a7b66b307 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.165298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] PATH_INFO: `/v3/projects/bacc28906e91432fb9902c57cc0b6e55/users/4a5df5d9b5ec44ef955a1f08718f96df/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.170554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacc28906e91432fb9902c57cc0b6e55, user_id=4a5df5d9b5ec44ef955a1f08718f96df, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.171173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf080e-e398-44cf-8675-5397dc2e5683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.172032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf080e-e398-44cf-8675-5397dc2e5683 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.172343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf080e-e398-44cf-8675-5397dc2e5683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.172624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf080e-e398-44cf-8675-5397dc2e5683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.175043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.175180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf080e-e398-44cf-8675-5397dc2e5683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.177774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf080e-e398-44cf-8675-5397dc2e5683 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.206105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 923/1881] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:16 2022] PUT /identity/v3/projects/bacc28906e91432fb9902c57cc0b6e55/users/4a5df5d9b5ec44ef955a1f08718f96df/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:16.207633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.228590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 959/1882] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:16.235054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.235288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.235545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.299669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] MFA Rules not processed for user `4a5df5d9b5ec44ef955a1f08718f96df`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:16.301877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:16.375529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7998-cb2d-428e-9ae5-2f96de7ab1f7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.392278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.392830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.393107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.397401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.400222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.406101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.489943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 924/1883] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:16 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:16.490888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.505560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 960/1884] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:16.510926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.537798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3f046-6d39-4a09-8e47-6180b21d9b11 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.589656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:16.597955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:16.675108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.675770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.676057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.676337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.677175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.678669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50888-7f51-4da9-8dc6-80243066db8d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.679538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.694943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.695518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.695857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.699774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.700656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.702466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.705391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 961/1885] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:16 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:16.706254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.708324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:16.727655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 925/1886] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:16.727655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.753871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45900f9-d57b-4276-8131-c2a9227f3d8d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.773053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.775085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.775584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.776058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.779727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.783046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.802531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 926/1887] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:16.803786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.809191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8ade-552c-468c-b54b-f9ee8a816910 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.816151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c9eac-c036-441a-a3ec-08576d6dc1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.817001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9eac-c036-441a-a3ec-08576d6dc1ba service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.817442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9eac-c036-441a-a3ec-08576d6dc1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.817919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9eac-c036-441a-a3ec-08576d6dc1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.822345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ac62-7a04-41a3-8a0c-2ba3d4fae948 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:16.831861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:16.833136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.833661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.834174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] PATH_INFO: `/v3/projects/4eaf202514f14528ab6b10aaa9683148/users/6be12a5343ed43ca82389948d905c697/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:16.858121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eaf202514f14528ab6b10aaa9683148, user_id=6be12a5343ed43ca82389948d905c697, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.861478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.898522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 927/1888] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:16 2022] PUT /identity/v3/projects/4eaf202514f14528ab6b10aaa9683148/users/6be12a5343ed43ca82389948d905c697/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:16.900373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:16.919589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c9eac-c036-441a-a3ec-08576d6dc1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:16.922008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c9eac-c036-441a-a3ec-08576d6dc1ba service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:16.941732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 962/1889] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:16.945335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636d0c3-640b-448b-bc0a-c2a09dfa59d8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:16.945909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636d0c3-640b-448b-bc0a-c2a09dfa59d8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:16.946588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636d0c3-640b-448b-bc0a-c2a09dfa59d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.026931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520ce1-791f-4a66-b372-6e360fd0c33d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.041349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.042017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.042313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.042589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.043375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.045043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.062334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 928/1890] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:17.069350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.071604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5636d0c3-640b-448b-bc0a-c2a09dfa59d8 None None] MFA Rules not processed for user `6be12a5343ed43ca82389948d905c697`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:17.073269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5636d0c3-640b-448b-bc0a-c2a09dfa59d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:17.084290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b28a9-282a-42dc-a401-401ae0af6f62 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.106963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.107400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.107648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.107959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] PATH_INFO: `/v3/projects/639a16e3c4f349d2b19f8f3b1d3dbcbd/users/bccbaf17f12d4281af809cd9f07ebe71/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.134969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] RBAC: Authorizing `identity:create_grant(project_id=639a16e3c4f349d2b19f8f3b1d3dbcbd, user_id=bccbaf17f12d4281af809cd9f07ebe71, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.137721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.162753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 929/1891] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:17 2022] PUT /identity/v3/projects/639a16e3c4f349d2b19f8f3b1d3dbcbd/users/bccbaf17f12d4281af809cd9f07ebe71/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:17.163985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.175017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:17.212805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 963/1892] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:17.216947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.217756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.218811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.257168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd4761-e259-4391-823f-8de907a1691b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.261491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.262040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.262340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.263079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.264545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:17.265778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] MFA Rules not processed for user `bccbaf17f12d4281af809cd9f07ebe71`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:17.267247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:17.281674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:17.351736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 964/1893] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:17.352774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.355043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.359923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.371902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 930/1894] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:17.372931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.374466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:17.389312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0ad28-c243-42ac-b8d5-6fb3cc7d197e service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.395242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.395853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.396137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.396421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.407420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.409106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.421991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 931/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:17.447570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1939d7c-96c8-4d02-bf8f-bbfe525b7298 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.448427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.453068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.453433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.453721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.453999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.455002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.456707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.466823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655c1e3-f73d-43d0-a397-daea2d645006 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.482357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.483418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.483934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.484310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] PATH_INFO: `/v3/users/bccbaf17f12d4281af809cd9f07ebe71` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.486314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccbaf17f12d4281af809cd9f07ebe71)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.490043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.500353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 965/1896] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:17 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:17.505105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.607516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccbaf17f12d4281af809cd9f07ebe71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:17.633744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccbaf17f12d4281af809cd9f07ebe71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:17.634404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccbaf17f12d4281af809cd9f07ebe71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:17.645808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c385e-1b34-4f72-9bf4-33e4ccf03d87 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.653646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.654450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.654904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.655284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.656123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.658571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.670533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccbaf17f12d4281af809cd9f07ebe71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:17.672085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:17.698673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a417aa1a-cc1a-409c-982a-7073dcc12ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccbaf17f12d4281af809cd9f07ebe71'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:17.705921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 966/1897] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:17 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:17.707140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.709933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:17.712372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 932/1898] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:17 2022] DELETE /identity/v3/users/bccbaf17f12d4281af809cd9f07ebe71 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:17.716253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.716547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.717023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.723842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e2613-bdab-4dfb-a133-615e7dc455b9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.739172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:17.740475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:17.803852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.804583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.805017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.805309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.806114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.808698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.822017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 967/1899] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:17.833772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:17.872426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f10644-378a-4509-8fe8-e57234ce7a30 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:17.886260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:17.887268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:17.887755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:17.888725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] PATH_INFO: `/v3/projects/e6a7a03fc76d42c4a322ea83c7a709d3/users/919917c12771453393d9367da596a4d9/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:17.889378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 933/1900] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:17.928958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a7a03fc76d42c4a322ea83c7a709d3, user_id=919917c12771453393d9367da596a4d9, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:17.942973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:17.987969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 968/1901] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:17 2022] PUT /identity/v3/projects/e6a7a03fc76d42c4a322ea83c7a709d3/users/919917c12771453393d9367da596a4d9/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:17.993752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:18.015955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:18.020163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:18.034190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa80353-2e14-4730-94c3-9ea3f88f5e63 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:18.076381 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:18.172085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e2aa5-f0aa-45f9-90ad-23c69aecbe6b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:18.178034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:18.178791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:18.179125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:18.179404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:18.180129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:18.181902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:18.189906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1253c58-7465-49b9-b629-0ae6eadeaf1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:18.190592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1253c58-7465-49b9-b629-0ae6eadeaf1d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:18.191057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1253c58-7465-49b9-b629-0ae6eadeaf1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:18.191401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1253c58-7465-49b9-b629-0ae6eadeaf1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:18.195256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 934/1902] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:18.201864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:18.212777 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1253c58-7465-49b9-b629-0ae6eadeaf1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:18.215149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1253c58-7465-49b9-b629-0ae6eadeaf1d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:18.217734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a092d9-dc77-4767-863d-7f64631035f6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:18.223317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:18.224162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:18.224495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:18.224884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] PATH_INFO: `/v3/projects/e6a7a03fc76d42c4a322ea83c7a709d3/users/919917c12771453393d9367da596a4d9/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:18.227127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a7a03fc76d42c4a322ea83c7a709d3, user_id=919917c12771453393d9367da596a4d9, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:18.229823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:18.233294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 969/1903] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:18.262434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 935/1904] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:18 2022] PUT /identity/v3/projects/e6a7a03fc76d42c4a322ea83c7a709d3/users/919917c12771453393d9367da596a4d9/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:18.281361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:18.282328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:18.283347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:18.335131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] MFA Rules not processed for user `919917c12771453393d9367da596a4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:18.337295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:18.462368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:18.493865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 970/1905] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:18.596356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:18.600225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:18.773151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98becad-e440-4bfd-a1da-abe735ac7e40 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:18.778077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:18.779016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:18.779700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:18.780430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:18.793701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:18.795831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:18.825180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 936/1906] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:19.051861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:19.076089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452299a-e6c5-4be5-8ba5-42a767ae9036 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:19.094878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:19.094878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.094878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.094878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.095507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:19.133196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:19.134957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:19.187163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:19.189353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9419a98-c7a6-4d7d-8839-cd2c3ebc9669 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:19.201675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 971/1907] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:19.256426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca4387-1117-4197-8196-98639278e30c service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:19.264068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:19.264835 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.265147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.265930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] PATH_INFO: `/v3/projects/639a16e3c4f349d2b19f8f3b1d3dbcbd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.266917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] RBAC: Authorizing `identity:delete_project(project_id=639a16e3c4f349d2b19f8f3b1d3dbcbd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:19.268874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:19.323692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:19.330155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639a16e3c4f349d2b19f8f3b1d3dbcbd', 'request_id': 'req-7dd02bd9-539e-467d-8b94-2cc36cf8470b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:19.342861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639a16e3c4f349d2b19f8f3b1d3dbcbd', 'request_id': 'req-7dd02bd9-539e-467d-8b94-2cc36cf8470b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:19.354977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639a16e3c4f349d2b19f8f3b1d3dbcbd', 'request_id': 'req-7dd02bd9-539e-467d-8b94-2cc36cf8470b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:19.370076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] The token cache is being invalidate because project 639a16e3c4f349d2b19f8f3b1d3dbcbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:19.370461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:19.371813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7dd02bd9-539e-467d-8b94-2cc36cf8470b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:19.373909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 937/1908] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:19 2022] DELETE /identity/v3/projects/639a16e3c4f349d2b19f8f3b1d3dbcbd => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:19.379355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012c80a-8158-4888-aac0-714771863950 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.379355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012c80a-8158-4888-aac0-714771863950 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.380077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012c80a-8158-4888-aac0-714771863950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.412120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c012c80a-8158-4888-aac0-714771863950 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:19.413968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c012c80a-8158-4888-aac0-714771863950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:19.530401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 972/1909] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:19.540631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.540926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.541258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.579394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:19.581359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:19.594666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 938/1910] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:19.599339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012c80a-8158-4888-aac0-714771863950 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:19.612888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012c80a-8158-4888-aac0-714771863950 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:19.619431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:19.620213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.620498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.620774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.621717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:19.622236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:19.624238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:19.645467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 973/1911] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:14:19.651447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:19.664544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cb4ae-0b16-42ec-bbf0-d518c88940aa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:19.678314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:19.679189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.682636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.683102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.685028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:19.687274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:19.752416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 939/1912] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:19 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:19.755396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:19.784058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b78a-ad7f-4cd3-a753-c27e22c384fc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:19.894427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:19.895853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.896307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.896853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.898243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:19.912817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:19.941687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 974/1913] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:19 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:19.947929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:19.964538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aae10-6923-44c1-8f25-2dcfa04f6099 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:19.972776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:19.973777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:19.974556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:19.975006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:19.976259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:19.980172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:19.997982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 940/1914] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:20.001177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.014393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848183e3-ce96-4433-8087-b34fe487b689 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.020417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221f8a-4fe2-4b69-bc33-d6557c6ad021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.021079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221f8a-4fe2-4b69-bc33-d6557c6ad021 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.021415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221f8a-4fe2-4b69-bc33-d6557c6ad021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.021708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221f8a-4fe2-4b69-bc33-d6557c6ad021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.022437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a221f8a-4fe2-4b69-bc33-d6557c6ad021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.024191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a221f8a-4fe2-4b69-bc33-d6557c6ad021 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.027132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:20.040270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 975/1915] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:20.044804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.057073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eb1af-0a3a-4343-9e59-a41bb030136b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.063215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.064151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.064603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.065077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] PATH_INFO: `/v3/projects/6775d899bb554dbd9eade8d97b675873/users/4c5f995c74964ceba6cd92de8744231c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.086247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6775d899bb554dbd9eade8d97b675873, user_id=4c5f995c74964ceba6cd92de8744231c, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.089276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.116706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 941/1916] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:20 2022] PUT /identity/v3/projects/6775d899bb554dbd9eade8d97b675873/users/4c5f995c74964ceba6cd92de8744231c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:20.120127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.120418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.120687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.160121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] MFA Rules not processed for user `4c5f995c74964ceba6cd92de8744231c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:20.161569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:20.287471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 976/1917] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:20.289292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.396053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c2d16-3834-40b1-8598-064cade24763 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.401172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.401829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.402112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.402382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.403102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.405032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.437102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 942/1918] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:20 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:20.441296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.460981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373aaea-ec6d-44fd-99c9-0854442ab69f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.525435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.526349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.526639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.527018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.527784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.529756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.555629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 977/1919] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:20 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:20.559767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.570472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc673e6-98cb-4f82-8ffc-38499478ee95 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.575250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.575928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.576236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.576533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.577215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.578897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.595151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 943/1920] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:20.599732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.615935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a6a9-4f4f-4963-9105-36415ada00ce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.624536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f5f91-c9ca-4554-afeb-86164be48461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.625250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f5f91-c9ca-4554-afeb-86164be48461 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.625541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f5f91-c9ca-4554-afeb-86164be48461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.625823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f5f91-c9ca-4554-afeb-86164be48461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.626562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f5f91-c9ca-4554-afeb-86164be48461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.628238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f5f91-c9ca-4554-afeb-86164be48461 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.640517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 978/1921] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:20.645333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.656619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46700-0e7e-4165-bd06-a36a964b5421 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.663140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.663981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.664332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.664704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] PATH_INFO: `/v3/projects/09c78aa6653c423f8710d85249e8630b/users/2069cded934642b8be4e8250fca0befe/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.683565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c78aa6653c423f8710d85249e8630b, user_id=2069cded934642b8be4e8250fca0befe, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.687486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:20.705403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 944/1922] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:20 2022] PUT /identity/v3/projects/09c78aa6653c423f8710d85249e8630b/users/2069cded934642b8be4e8250fca0befe/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:20.716564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.716867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.717184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.751300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] MFA Rules not processed for user `2069cded934642b8be4e8250fca0befe`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:20.753012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:20.844173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:20.858789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 979/1923] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:20.863599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:20.899271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:20.957391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4922e5d-5101-4d85-8f8d-647e73bc7b00 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:20.968762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:20.969773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:20.970109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:20.971116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:20.971991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:20.974010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.019741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 945/1924] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:20 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:21.024026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.042112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405e77-840f-4798-b1de-c9412f5f5264 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.137893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815d836-162c-47fb-8542-910391845965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.138650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815d836-162c-47fb-8542-910391845965 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.139167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815d836-162c-47fb-8542-910391845965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.139808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815d836-162c-47fb-8542-910391845965 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.140753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815d836-162c-47fb-8542-910391845965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.143551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815d836-162c-47fb-8542-910391845965 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.183225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 980/1925] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:21 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:21.183843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.202923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096870f-785c-4dbc-9cbd-fef002cfe55b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.217374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.218248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.218613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.219264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.219851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.222130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.253793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 946/1926] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:21.258911 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815d836-162c-47fb-8542-910391845965 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.286594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.292451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:21.295151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815d836-162c-47fb-8542-910391845965 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.311625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.312445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.312799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.313148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] PATH_INFO: `/v3/projects/21e50cc16ade4b278a0249a6e19267e7/users/f2e88a17e1594c6fb157222753387fba/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.315920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ffab-f4bf-4128-b998-65f114ba3e25 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.322199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.322946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.323302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.323648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] PATH_INFO: `/v3/users/b47d3668a8ca4adfb484c423fc08e0e5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.324948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] RBAC: Authorizing `identity:delete_user(user_id=b47d3668a8ca4adfb484c423fc08e0e5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.327072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.338155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e50cc16ade4b278a0249a6e19267e7, user_id=f2e88a17e1594c6fb157222753387fba, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.341068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.363140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 981/1927] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:21 2022] PUT /identity/v3/projects/21e50cc16ade4b278a0249a6e19267e7/users/f2e88a17e1594c6fb157222753387fba/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:21.368172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.383862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b47d3668a8ca4adfb484c423fc08e0e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:21.393316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b47d3668a8ca4adfb484c423fc08e0e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:21.394100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b47d3668a8ca4adfb484c423fc08e0e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:21.410330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b47d3668a8ca4adfb484c423fc08e0e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:21.425763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2f38b6f8-1fc7-4af3-9f95-d6c3ef8663de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b47d3668a8ca4adfb484c423fc08e0e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:21.433031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 947/1928] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:21 2022] DELETE /identity/v3/users/b47d3668a8ca4adfb484c423fc08e0e5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:21.438955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.439664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.440145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.441531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff827284-b129-4947-8976-6ba31d016280 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.485247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:21.487004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:21.540109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.540700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.541010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.541287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.542123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.544027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.566675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 982/1929] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:21.571365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.576197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:21.598267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 948/1930] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:21.600238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:21.606958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196ece1-0d2e-4b48-bd03-7f298d8acf80 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.615048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.617981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:21.620052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3c368-ef05-4f59-a9a2-f9e32fed163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.620985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3c368-ef05-4f59-a9a2-f9e32fed163e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.621453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3c368-ef05-4f59-a9a2-f9e32fed163e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.621874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3c368-ef05-4f59-a9a2-f9e32fed163e admin admin] PATH_INFO: `/v3/projects/21e50cc16ade4b278a0249a6e19267e7/users/f2e88a17e1594c6fb157222753387fba/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.624472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3c368-ef05-4f59-a9a2-f9e32fed163e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e50cc16ade4b278a0249a6e19267e7, user_id=f2e88a17e1594c6fb157222753387fba, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.627787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3c368-ef05-4f59-a9a2-f9e32fed163e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.644239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:21.658870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 983/1931] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:21 2022] PUT /identity/v3/projects/21e50cc16ade4b278a0249a6e19267e7/users/f2e88a17e1594c6fb157222753387fba/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:21.667143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.667430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.667709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.708294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] MFA Rules not processed for user `f2e88a17e1594c6fb157222753387fba`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:21.710668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:21.725723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fcf7c-fd71-428d-a1e6-859aa14adb75 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.829571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 984/1932] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:21.839975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.840750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.841133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.841498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.845230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:21.849231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:21.935049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:21.936657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:21.950787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 949/1933] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:21.962932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a6e97-fd5c-4fda-82a1-efedfbb078bd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:21.967221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:21.967952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:21.968441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:21.968923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:21.971574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.050432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:22.052056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:22.063657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 985/1934] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:22.170461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:22.172096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.189792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44463707-6da6-4446-b89a-1f9cbf383a29 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:22.196830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:22.197518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:22.198529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:22.198817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] PATH_INFO: `/v3/users/6be12a5343ed43ca82389948d905c697` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:22.201778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be12a5343ed43ca82389948d905c697)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:22.204421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:22.276776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be12a5343ed43ca82389948d905c697'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.292538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be12a5343ed43ca82389948d905c697'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.304695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be12a5343ed43ca82389948d905c697'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.319259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:22.332487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be12a5343ed43ca82389948d905c697'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.344553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d91763e1-f3e3-4b72-b521-ce4f70bdb315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be12a5343ed43ca82389948d905c697'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.358441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:22.360577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.364503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 950/1935] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:22 2022] DELETE /identity/v3/users/6be12a5343ed43ca82389948d905c697 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:22.369186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:22.370086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:22.370812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:22.388211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a58b9-695f-4508-ac09-f9eed78bc2b2 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:22.411500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:22.413382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.512661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:22.576660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 951/1936] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:22.585188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:22.585890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:22.586276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:22.586624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] PATH_INFO: `/v3/projects/d93e7712e88744d1a5d6bbe9b305ce66` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:22.587958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93e7712e88744d1a5d6bbe9b305ce66)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:22.590287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:22.604536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:22.607191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.695955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93e7712e88744d1a5d6bbe9b305ce66', 'request_id': 'req-3a970127-9904-497c-8a53-c064a8bb62b1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.703463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93e7712e88744d1a5d6bbe9b305ce66', 'request_id': 'req-3a970127-9904-497c-8a53-c064a8bb62b1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.717267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93e7712e88744d1a5d6bbe9b305ce66', 'request_id': 'req-3a970127-9904-497c-8a53-c064a8bb62b1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.732597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d43f-e548-4032-b750-09ac0e99125e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:22.764399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] The token cache is being invalidate because project d93e7712e88744d1a5d6bbe9b305ce66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:22.764885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.765643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:22.768863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 986/1937] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:22 2022] DELETE /identity/v3/projects/d93e7712e88744d1a5d6bbe9b305ce66 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:22.773172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:22.775141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.805226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:22.806178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:22.806525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:22.807038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:22.809560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:22.904215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:22.943232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970127-9904-497c-8a53-c064a8bb62b1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:22.949274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:22.951448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:22.954152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:22.955100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:22.955533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:22.955960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] PATH_INFO: `/v3/users/4a5df5d9b5ec44ef955a1f08718f96df` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:22.961483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a5df5d9b5ec44ef955a1f08718f96df)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:22.963859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:22.974323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 952/1938] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:23.041864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a5df5d9b5ec44ef955a1f08718f96df'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.042600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a5df5d9b5ec44ef955a1f08718f96df'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.044578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a5df5d9b5ec44ef955a1f08718f96df'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.060635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a5df5d9b5ec44ef955a1f08718f96df'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.074297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:23.099355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a5df5d9b5ec44ef955a1f08718f96df'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.130415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 987/1939] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:22 2022] DELETE /identity/v3/users/4a5df5d9b5ec44ef955a1f08718f96df => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:23.491556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:23.502874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e66518-1fb3-4218-9d35-081ef0d53251 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:23.507813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:23.512087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:23.583193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:23.583806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:23.584360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:23.584837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] PATH_INFO: `/v3/projects/bacc28906e91432fb9902c57cc0b6e55` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:23.586225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bacc28906e91432fb9902c57cc0b6e55)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:23.588420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:23.615538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25037736-238d-419a-8bed-ae30384434f7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:23.621929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:23.622825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:23.623199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:23.623487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] PATH_INFO: `/v3/projects/4eaf202514f14528ab6b10aaa9683148` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:23.626469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eaf202514f14528ab6b10aaa9683148)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:23.628984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:23.654925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bacc28906e91432fb9902c57cc0b6e55', 'request_id': 'req-a86b73f1-6a78-4cff-a433-d1db7ab80c61'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.671139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bacc28906e91432fb9902c57cc0b6e55', 'request_id': 'req-a86b73f1-6a78-4cff-a433-d1db7ab80c61'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.684830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bacc28906e91432fb9902c57cc0b6e55', 'request_id': 'req-a86b73f1-6a78-4cff-a433-d1db7ab80c61'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.697476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] The token cache is being invalidate because project bacc28906e91432fb9902c57cc0b6e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:23.697779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.698457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a86b73f1-6a78-4cff-a433-d1db7ab80c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.702935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 953/1940] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:23 2022] DELETE /identity/v3/projects/bacc28906e91432fb9902c57cc0b6e55 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:23.705414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf450c-179b-4bc2-94cb-8d6ae204a260 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:23.705708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf450c-179b-4bc2-94cb-8d6ae204a260 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:23.706131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf450c-179b-4bc2-94cb-8d6ae204a260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:23.719964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eaf202514f14528ab6b10aaa9683148', 'request_id': 'req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.730902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eaf202514f14528ab6b10aaa9683148', 'request_id': 'req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.741922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eaf202514f14528ab6b10aaa9683148', 'request_id': 'req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.753758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-77cf450c-179b-4bc2-94cb-8d6ae204a260 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:23.755454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-77cf450c-179b-4bc2-94cb-8d6ae204a260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:23.776925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] The token cache is being invalidate because project 4eaf202514f14528ab6b10aaa9683148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:23.776925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.776925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.776925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 988/1941] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:23 2022] DELETE /identity/v3/projects/4eaf202514f14528ab6b10aaa9683148 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:23.781442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:23.783411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:23.861232 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 954/1942] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:23.874595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:23.875386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:23.876034 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:23.885148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84979bb-45e9-4b0c-8a0f-2cdfcfffed99 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:23.897288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:23.898736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:23.899543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:23.900280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] PATH_INFO: `/v3/users/919917c12771453393d9367da596a4d9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:23.902784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:23.904123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:23.905373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=919917c12771453393d9367da596a4d9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:23.909814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:23.913001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 955/1943] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:23.935427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:23.936701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:23.948335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187fc9-932f-4d33-9bf4-a35c0a85cfd5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:23.956183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:23.957418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:23.957418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:23.957784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:23.958440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:23.958762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:23.960599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:23.973329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:23.982372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 956/1944] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:23.984569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:23.988677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:23.996791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919917c12771453393d9367da596a4d9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.997496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919917c12771453393d9367da596a4d9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:23.998186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919917c12771453393d9367da596a4d9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:24.013640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919917c12771453393d9367da596a4d9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:24.032322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919917c12771453393d9367da596a4d9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:24.063620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 989/1945] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:23 2022] DELETE /identity/v3/users/919917c12771453393d9367da596a4d9 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:24.063620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:24.065102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:24.177208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b6ae6-2a00-459b-bf0f-473465b1d846 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:24.305996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84278a28-18e6-4662-a232-1f7523f5fa5d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:24.319118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:24.320402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:24.324447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:24.325087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:24.326384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:24.335991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:24.390736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:24.392319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:24.392902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:24.393460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:24.407813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:24.488355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 990/1946] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:24 2022] POST /identity/v3/projects => generated 387 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:24.489199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:24.568941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:24.573632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:24.604260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 957/1947] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:24.648056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a4bca-0a76-4f3b-a023-eaab66bbd656 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:24.656829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:24.659251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:24.659814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:24.660316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:24.661554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:24.666773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:24.725239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 991/1948] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:24 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:24.731335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:24.747309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a77e0f-b0b7-444e-ab4f-dd270078269b service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:24.760767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:24.761467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:24.761757 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:24.762039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:24.763072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:24.767096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:24.790508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 958/1949] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:24.796344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:24.804577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:24.811598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc88b-6b61-4954-94de-cc527f76ed78 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:24.831328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d1e25-3b71-4e08-b47d-dfde9914b5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:24.832138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1e25-3b71-4e08-b47d-dfde9914b5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:24.832456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1e25-3b71-4e08-b47d-dfde9914b5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:24.832743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1e25-3b71-4e08-b47d-dfde9914b5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:24.833747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1e25-3b71-4e08-b47d-dfde9914b5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:24.835579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1e25-3b71-4e08-b47d-dfde9914b5c0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:24.890089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 992/1950] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:24.903958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:24.915658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0191fc4-d039-4c68-8875-f604fae9549c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:24.920953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:24.921659 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:24.921951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:24.922235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] PATH_INFO: `/v3/projects/209718226e80474bbdc5bb4104ddeff3/users/4f351a3b356c4f4a84a7089f5b1b2428/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:24.947579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209718226e80474bbdc5bb4104ddeff3, user_id=4f351a3b356c4f4a84a7089f5b1b2428, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:24.955002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:24.989943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 959/1951] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:24 2022] PUT /identity/v3/projects/209718226e80474bbdc5bb4104ddeff3/users/4f351a3b356c4f4a84a7089f5b1b2428/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:25.008549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.008821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.009445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.083123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] MFA Rules not processed for user `4f351a3b356c4f4a84a7089f5b1b2428`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:25.087851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:25.201960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.205129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:25.276856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 993/1952] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:25.283427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.340110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:25.432242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2fc1-cea5-4f72-9cf2-834aa66d6ea4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.443993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.445232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.445697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.446169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.447367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.450539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.475358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3cf98-4c6a-42ed-86cf-89bd7fe4cf08 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.489397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.492544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.492809 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.493390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] PATH_INFO: `/v3/projects/e6a7a03fc76d42c4a322ea83c7a709d3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.496743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a7a03fc76d42c4a322ea83c7a709d3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.502304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.525945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 994/1953] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:25 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:25.527265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.563452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a7a03fc76d42c4a322ea83c7a709d3', 'request_id': 'req-843bf9f9-d449-4dff-ba75-37131e9033c4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:25.573769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a7a03fc76d42c4a322ea83c7a709d3', 'request_id': 'req-843bf9f9-d449-4dff-ba75-37131e9033c4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:25.580485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a7a03fc76d42c4a322ea83c7a709d3', 'request_id': 'req-843bf9f9-d449-4dff-ba75-37131e9033c4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:25.596364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] The token cache is being invalidate because project e6a7a03fc76d42c4a322ea83c7a709d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:25.596957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:25.598189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-843bf9f9-d449-4dff-ba75-37131e9033c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:25.604102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 960/1954] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:25 2022] DELETE /identity/v3/projects/e6a7a03fc76d42c4a322ea83c7a709d3 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:25.607557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9c448-2d53-482c-b8cb-b03da9f53b63 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.608068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9c448-2d53-482c-b8cb-b03da9f53b63 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.608718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9c448-2d53-482c-b8cb-b03da9f53b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.624715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b3bc-b0ba-47a3-acad-3427f0079327 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.638562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:25.650517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-15f9c448-2d53-482c-b8cb-b03da9f53b63 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:25.652070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-15f9c448-2d53-482c-b8cb-b03da9f53b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:25.693408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.694270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.694415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.694655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.695494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.697620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.732077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 995/1955] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:25 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:25.734989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.737677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:25.756272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 961/1956] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:25.758266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.758550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.758855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.759310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8949a1-9c9d-4832-a13e-808d68a32822 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.764513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.765390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.766068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.766499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.767875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.771299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.785270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 996/1957] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:25.788732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:25.789285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.790403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:25.803002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 962/1958] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:25.805684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.811692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042351-68de-4f49-8def-3ece0951a7d6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.817830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.819097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0a1bb-303a-490f-ba5b-f64552bd5152 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.820610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.820898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.821178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.821970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.823752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.829233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.830081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.830795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.831168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.832102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:25.832542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.834383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.839077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 997/1959] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:25.843306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.845625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:25.851363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 963/1960] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:14:25.853065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.866316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce31a-acb1-4211-b27f-d0cb23b55f32 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.871289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:25.872104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:25.872307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:25.872539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] PATH_INFO: `/v3/projects/bace7afa706242e59685f6b1cbd17d6b/users/16c95ced769b4ee7b23e0a4ac46d5c4c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:25.892057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bace7afa706242e59685f6b1cbd17d6b, user_id=16c95ced769b4ee7b23e0a4ac46d5c4c, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:25.896118 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:25.938873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 964/1961] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:25 2022] PUT /identity/v3/projects/bace7afa706242e59685f6b1cbd17d6b/users/16c95ced769b4ee7b23e0a4ac46d5c4c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:25.938873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:25.939198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf023f2-d953-41f5-b560-4f415a4b1b58 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:25.983862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:26.007049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.008042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.008482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.008762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.010782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:26.049596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079962-382b-49a4-8e2c-36c2756adb09 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.054778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794ec3a-a5fb-4025-be67-15c0d262b224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.055451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ec3a-a5fb-4025-be67-15c0d262b224 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.056387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ec3a-a5fb-4025-be67-15c0d262b224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.056387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ec3a-a5fb-4025-be67-15c0d262b224 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.056677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794ec3a-a5fb-4025-be67-15c0d262b224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.058596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794ec3a-a5fb-4025-be67-15c0d262b224 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.085631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:26.098606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 965/1962] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:25 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:26.106789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.108305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.109303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.109615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.109898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.122703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 998/1963] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:26.123812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.140071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a454ed9-5444-4342-8822-c75270bfaaad service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.164023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] MFA Rules not processed for user `16c95ced769b4ee7b23e0a4ac46d5c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:26.165587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:26.228259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.229212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.229571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.229888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.232064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.234584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.267789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 999/1964] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:26.272746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 966/1965] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:26.273696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.277273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.283986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971311e3-3f76-40ca-9f7a-4390676e4e0d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.288919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a912c0-8be9-4b73-9b94-0c48cde03218 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.289239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.289944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.290343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.290672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.291426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.293024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.293749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.294376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.294658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.297090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.297090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.297740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.306789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1000/1966] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:26.310477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.323653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11d221-30b2-4a35-9ef8-b06bb3e68551 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.331103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 967/1967] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:26.335530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.335775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.336619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.337118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.337370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.338224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.340094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.351863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1001/1968] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:26.356417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.368163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f523-0b7e-4df5-931b-3b61eba76159 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.421378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd1ac-45af-4df0-8a45-b7f7ab52c4d3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.427203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.428061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.428337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.428978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.429533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.432204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.442121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d420e-6e99-4588-bbb8-8c96388220a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.443753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d420e-6e99-4588-bbb8-8c96388220a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.444077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d420e-6e99-4588-bbb8-8c96388220a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.444413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d420e-6e99-4588-bbb8-8c96388220a4 admin admin] PATH_INFO: `/v3/projects/6e9ed216d2b54bde8b33137832a41554/users/cc6d1b768c4549dbb66ad771f7a07135/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.460606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 968/1969] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:26.464319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.467225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d420e-6e99-4588-bbb8-8c96388220a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9ed216d2b54bde8b33137832a41554, user_id=cc6d1b768c4549dbb66ad771f7a07135, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.469836 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d420e-6e99-4588-bbb8-8c96388220a4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.475822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:26.481224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c55af-d0f8-491f-b718-149221ebe977 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.486368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1002/1970] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:26 2022] PUT /identity/v3/projects/6e9ed216d2b54bde8b33137832a41554/users/cc6d1b768c4549dbb66ad771f7a07135/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:26.501665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.502145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.502452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.534774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] MFA Rules not processed for user `cc6d1b768c4549dbb66ad771f7a07135`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:26.536047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:26.562277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.563296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.563596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.563876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.564676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.566337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.570120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:26.577667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 969/1971] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:26.581903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.594387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a8b7b-8cad-4bf8-b408-7d5b9a95f7b6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.599261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.599933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.600216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.600490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] PATH_INFO: `/v3/projects/6dec8819ba2744d199765e4e30a874ef/users/8d7348070722415d90e416fdd2781dbd/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.608645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1003/1972] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:26.613477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.615206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dec8819ba2744d199765e4e30a874ef, user_id=8d7348070722415d90e416fdd2781dbd, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.617967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.624373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c7cf6-2cce-486d-a90e-afd9509d1087 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.633375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.634018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.634329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.634575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.635295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.636984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 970/1973] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:26 2022] PUT /identity/v3/projects/6dec8819ba2744d199765e4e30a874ef/users/8d7348070722415d90e416fdd2781dbd/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:26.639044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.640937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.670036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1004/1974] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:26.673966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.711783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b099e-a5af-4cc6-b9b9-a03260fad337 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.737910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779a4b4-412e-4064-868c-e42fd574294b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.742819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.743521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.743786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.744083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.744789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.746732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.770157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1005/1975] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:26.775354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.777501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.778154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.778494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.778745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.779655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.781271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.788878 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce8104-1204-46b0-adde-ee7753acff60 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.797420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 971/1976] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:26.797603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.797603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.797603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.797877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.798545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.798644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.800246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.809662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081be036-5ebe-45f9-866d-88b1b34f2db1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.811532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1006/1977] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:26.814649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82320e0d-a1a0-4575-811d-604bb359ef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.815382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320e0d-a1a0-4575-811d-604bb359ef0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.815604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.815717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320e0d-a1a0-4575-811d-604bb359ef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.816093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320e0d-a1a0-4575-811d-604bb359ef0b admin admin] PATH_INFO: `/v3/projects/6dec8819ba2744d199765e4e30a874ef/users/8d7348070722415d90e416fdd2781dbd/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.818078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82320e0d-a1a0-4575-811d-604bb359ef0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dec8819ba2744d199765e4e30a874ef, user_id=8d7348070722415d90e416fdd2781dbd, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.820830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82320e0d-a1a0-4575-811d-604bb359ef0b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.825774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf92079-e6aa-45d2-82b7-ad129a230f09 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:26.844643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 972/1978] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:26 2022] PUT /identity/v3/projects/6dec8819ba2744d199765e4e30a874ef/users/8d7348070722415d90e416fdd2781dbd/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:26.846550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.846858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.847327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.865778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:26.882099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] MFA Rules not processed for user `8d7348070722415d90e416fdd2781dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:26.883370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:26.892916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:26.893616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:26.893925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:26.894190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] PATH_INFO: `/v3/projects/b9349ff259b047609111ddae54c192ad/users/cd0b642f50a9432190fe29476be52301/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:26.910778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9349ff259b047609111ddae54c192ad, user_id=cd0b642f50a9432190fe29476be52301, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:26.913641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:26.937473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1007/1979] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:26 2022] PUT /identity/v3/projects/b9349ff259b047609111ddae54c192ad/users/cd0b642f50a9432190fe29476be52301/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:26.938711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:26.955631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:26.970013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 973/1980] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:26.978964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:27.036351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b1ea0-34b0-4f81-9f82-4b4e2bdd4f7b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:27.044561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb803a3-9678-4f91-9440-ff3e47702851 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:27.049908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:27.051123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:27.051519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:27.051976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:27.053051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:27.054895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:27.071490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1008/1981] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:27.076318 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:27.088767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ce6bb-659d-4031-bf86-71c5bee0a99a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:27.094155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e32f-6895-4792-8e73-e73511af0742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:27.094981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e32f-6895-4792-8e73-e73511af0742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:27.095248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e32f-6895-4792-8e73-e73511af0742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:27.095470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e32f-6895-4792-8e73-e73511af0742 admin admin] PATH_INFO: `/v3/projects/b9349ff259b047609111ddae54c192ad/users/cd0b642f50a9432190fe29476be52301/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:27.097458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:27.097590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e32f-6895-4792-8e73-e73511af0742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9349ff259b047609111ddae54c192ad, user_id=cd0b642f50a9432190fe29476be52301, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:27.098175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:27.098521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:27.098885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:27.100480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e32f-6895-4792-8e73-e73511af0742 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:27.112116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1009/1982] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:27 2022] PUT /identity/v3/projects/b9349ff259b047609111ddae54c192ad/users/cd0b642f50a9432190fe29476be52301/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:27.123720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:27.124054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:27.124284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:27.157041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] MFA Rules not processed for user `cd0b642f50a9432190fe29476be52301`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:27.158354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:27.208800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:27.211455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:27.216128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:27.224926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 974/1983] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:27.326569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1010/1984] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:27.434186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:27.447220 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee93d9b-af5e-4240-a762-f39b1138ce2e service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:27.504571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8759005-3bd9-4ad8-b5a7-9290dccb797b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:27.505654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8759005-3bd9-4ad8-b5a7-9290dccb797b service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:27.506005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8759005-3bd9-4ad8-b5a7-9290dccb797b service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:27.506280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8759005-3bd9-4ad8-b5a7-9290dccb797b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:27.519495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8759005-3bd9-4ad8-b5a7-9290dccb797b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:27.521255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8759005-3bd9-4ad8-b5a7-9290dccb797b service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:27.532370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 975/1985] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:27.999614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:28.094263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d749e0-6248-4145-b94b-21f75655401e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:28.099682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:28.100374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:28.100621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:28.100899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] PATH_INFO: `/v3/users/cc6d1b768c4549dbb66ad771f7a07135` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:28.102148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6d1b768c4549dbb66ad771f7a07135)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:28.104148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:28.126677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:28.161636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6d1b768c4549dbb66ad771f7a07135'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:28.162410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6d1b768c4549dbb66ad771f7a07135'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:28.164035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6d1b768c4549dbb66ad771f7a07135'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:28.172166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6d1b768c4549dbb66ad771f7a07135'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:28.188134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6d1b768c4549dbb66ad771f7a07135'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:28.206240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1011/1986] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:27 2022] DELETE /identity/v3/users/cc6d1b768c4549dbb66ad771f7a07135 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:28.207845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:28.208358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:28.208510 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:28.223724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:28.225323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:28.228822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:28.230017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:28.302819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c24c4-bf28-4d9a-b03a-1ef262bb6590 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:28.308436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:28.309333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:28.309761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:28.310180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:28.312527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:28.319037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 976/1987] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:28.334024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:28.336842 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:28.432269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:28.434372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:28.442176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664067e5-54f6-43f0-82fc-438923c16f28 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:28.447999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:28.448786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:28.449055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:28.449322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:28.453461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1012/1988] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:28.465730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:28.467506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:28.480904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 977/1989] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:28.981177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:28.983145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:29.028451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55a951-363f-427a-8282-9fe5f914dd2f service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:29.036381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:29.037490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:29.038288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:29.038707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] PATH_INFO: `/v3/projects/6e9ed216d2b54bde8b33137832a41554` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:29.041540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9ed216d2b54bde8b33137832a41554)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:29.045303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:29.111136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:29.119065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9ed216d2b54bde8b33137832a41554', 'request_id': 'req-e0ba9580-eea3-41b9-b581-c5d02d33243b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.132479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9ed216d2b54bde8b33137832a41554', 'request_id': 'req-e0ba9580-eea3-41b9-b581-c5d02d33243b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.141987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9ed216d2b54bde8b33137832a41554', 'request_id': 'req-e0ba9580-eea3-41b9-b581-c5d02d33243b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.160394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] The token cache is being invalidate because project 6e9ed216d2b54bde8b33137832a41554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:29.160394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.160394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.160394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1013/1990] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:28 2022] DELETE /identity/v3/projects/6e9ed216d2b54bde8b33137832a41554 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:29.163296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:29.175024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90dad6-d7bb-4a0b-89f0-ab762f1b4528 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:29.231562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:29.249369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:29.250536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:29.250536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:29.250797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] PATH_INFO: `/v3/users/cd0b642f50a9432190fe29476be52301` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:29.252324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd0b642f50a9432190fe29476be52301)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:29.254399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:29.322310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd0b642f50a9432190fe29476be52301'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.331109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd0b642f50a9432190fe29476be52301'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.332009 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd0b642f50a9432190fe29476be52301'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.350260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd0b642f50a9432190fe29476be52301'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.369350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57612175-7cd7-41dd-b0c1-1cff1a213979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd0b642f50a9432190fe29476be52301'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.381935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 978/1991] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:29 2022] DELETE /identity/v3/users/cd0b642f50a9432190fe29476be52301 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:29.401499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:29.403145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:29.422520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba9580-eea3-41b9-b581-c5d02d33243b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:29.495379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:29.495722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:29.495935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:29.496254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] PATH_INFO: `/v3/users/a665f3e557e442ed990d56161749f0c1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:29.497690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a665f3e557e442ed990d56161749f0c1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:29.499957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:29.590302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a665f3e557e442ed990d56161749f0c1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.591137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a665f3e557e442ed990d56161749f0c1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.595215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a665f3e557e442ed990d56161749f0c1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.613195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a665f3e557e442ed990d56161749f0c1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.637737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a665f3e557e442ed990d56161749f0c1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:29.660770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1014/1992] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:29 2022] DELETE /identity/v3/users/a665f3e557e442ed990d56161749f0c1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:29.665715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:29.666010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:29.666296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:29.723066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:29.730202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:29.731618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:29.737627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:29.856380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:29.872605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 979/1993] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:29.878106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6343-5316-4f33-82f3-0b936c3a1c52 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:29.886214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:29.887704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:29.895909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:29.897717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:29.898008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:29.898293 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] PATH_INFO: `/v3/projects/b9349ff259b047609111ddae54c192ad` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:29.902942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9349ff259b047609111ddae54c192ad)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:29.907836 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:29.995657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9349ff259b047609111ddae54c192ad', 'request_id': 'req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.002658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1eee-7404-42b6-b74e-af88871c5c01 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.008611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9349ff259b047609111ddae54c192ad', 'request_id': 'req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.022874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9349ff259b047609111ddae54c192ad', 'request_id': 'req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.028942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:30.046062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] The token cache is being invalidate because project b9349ff259b047609111ddae54c192ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:30.046601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.047380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c77fd38c-c3ce-4132-9e65-4c6a9a72f3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.051495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1015/1994] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:29 2022] DELETE /identity/v3/projects/b9349ff259b047609111ddae54c192ad => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:30.058829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ca96-1810-4408-bd18-693576cf2901 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.059487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ca96-1810-4408-bd18-693576cf2901 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.059785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ca96-1810-4408-bd18-693576cf2901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.106280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.108282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.109163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.109437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.116209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.125713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-82f1ca96-1810-4408-bd18-693576cf2901 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:30.127023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-82f1ca96-1810-4408-bd18-693576cf2901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.202973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1016/1995] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:30.212137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.212489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.212781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.241874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:30.243184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.255520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1017/1996] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:30.266901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.274578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89f450-aa7e-4973-baa8-4ce7a9d93eb2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.281904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.282871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.283419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.283850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.285295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:30.285765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.288020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.308005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1018/1997] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:30.309012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.322217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.323805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.329249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c156ae2-1a8d-4e2a-a655-8eefe4d2e26d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.347965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.348633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.349025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.349303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.351615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.362960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 980/1998] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:30.363266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.405857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1019/1999] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:30 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:30.406297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.425270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7863d-a87d-442b-a998-4e4e90c59468 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.442172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.451876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.519457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:30.550415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf3e95-3029-47ba-abe2-01c774f48731 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.555898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.556586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.556872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.557150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] PATH_INFO: `/v3/users/4c5f995c74964ceba6cd92de8744231c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.558467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c5f995c74964ceba6cd92de8744231c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.560791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.570786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.571474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.571761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.572070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.572726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.575040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.605111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 981/2000] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:30 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:30.609371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.622979 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b192b0-e08d-49e7-b892-d2f0add910ae admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.626406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c5f995c74964ceba6cd92de8744231c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.627180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c5f995c74964ceba6cd92de8744231c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.627898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c5f995c74964ceba6cd92de8744231c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.630303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.631106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.631444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.631742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.632530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.634376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.638513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c5f995c74964ceba6cd92de8744231c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.652227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 982/2001] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:30.653548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.655564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.662440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cc601939-6a3e-433f-ba6f-43f5be33d279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c5f995c74964ceba6cd92de8744231c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:30.677421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a5c5d-6b26-48cc-aba1-88ed11fff225 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.679623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1020/2002] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:30 2022] DELETE /identity/v3/users/4c5f995c74964ceba6cd92de8744231c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:30.686027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.686460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.686633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.710933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:30.712201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.757923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:30.776934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.777759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.778042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.778319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.779226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.781368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.788636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1021/2003] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:30.799571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.802408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.814992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 983/2004] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:30.816687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:30.831038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3a7fc-30a8-499c-a4b8-0637304bda79 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.837723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502ba97-9480-47e9-9a1d-5f07b2814a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.838996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502ba97-9480-47e9-9a1d-5f07b2814a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.839477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502ba97-9480-47e9-9a1d-5f07b2814a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.839986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502ba97-9480-47e9-9a1d-5f07b2814a84 admin admin] PATH_INFO: `/v3/projects/9fa144b52443488fb36d21ed2d5dba86/users/712cfcc2adb4444795b0023048e3a6f0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.868436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502ba97-9480-47e9-9a1d-5f07b2814a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa144b52443488fb36d21ed2d5dba86, user_id=712cfcc2adb4444795b0023048e3a6f0, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:30.868988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502ba97-9480-47e9-9a1d-5f07b2814a84 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:30.895811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 984/2005] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:30 2022] PUT /identity/v3/projects/9fa144b52443488fb36d21ed2d5dba86/users/712cfcc2adb4444795b0023048e3a6f0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:30.905364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09fa6f-85de-4dc1-9ca5-48c4f8ef08c4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:30.906422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.906783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.907151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:30.922283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:30.947363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] MFA Rules not processed for user `712cfcc2adb4444795b0023048e3a6f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:30.948685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:30.982037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:30.983444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:30.984158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:30.984544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.051917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 985/2006] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:31.067143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:31.069072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:31.109521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:31.112228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:31.127529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1022/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:31.177784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f6346-4f2c-4c5c-a0ed-82549064cc31 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:31.184693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de04e97-56a9-43ed-bcb3-11444022638d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:31.185360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de04e97-56a9-43ed-bcb3-11444022638d service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:31.185702 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de04e97-56a9-43ed-bcb3-11444022638d service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:31.185986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de04e97-56a9-43ed-bcb3-11444022638d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.196597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de04e97-56a9-43ed-bcb3-11444022638d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:31.198057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de04e97-56a9-43ed-bcb3-11444022638d service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:31.209876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 986/2008] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:31.235965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:31.253364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007fc90-cf14-46f4-b905-5026f1205b43 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:31.264583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:31.265715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:31.266114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:31.266492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] PATH_INFO: `/v3/users/712cfcc2adb4444795b0023048e3a6f0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.270667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=712cfcc2adb4444795b0023048e3a6f0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:31.272704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:31.363235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712cfcc2adb4444795b0023048e3a6f0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.364259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712cfcc2adb4444795b0023048e3a6f0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.364883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712cfcc2adb4444795b0023048e3a6f0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.384369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712cfcc2adb4444795b0023048e3a6f0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.409227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712cfcc2adb4444795b0023048e3a6f0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.437554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1023/2009] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:31 2022] DELETE /identity/v3/users/712cfcc2adb4444795b0023048e3a6f0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:31.443020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:31.443613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:31.443698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.467988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:31.469235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:31.539124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:31.562336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 987/2010] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:31.578238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:31.581547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:31.657306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:31.696311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac81c62-3abf-4c7c-9184-c8bc2b79e562 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:31.702298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:31.703278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:31.703784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:31.704237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.716820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:31.718644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:31.757933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1024/2011] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:31.841144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:31.842908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:31.848651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:31.850880 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:31.868655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46955a-a0e2-4617-89e0-09d06a8bf781 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:31.873877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:31.874613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:31.874936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:31.875071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9a74d-def8-425b-8c8b-05314a2b6295 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:31.875265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] PATH_INFO: `/v3/projects/6775d899bb554dbd9eade8d97b675873` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.877096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6775d899bb554dbd9eade8d97b675873)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:31.879626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:31.897242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:31.899912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:31.900440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:31.900953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] PATH_INFO: `/v3/projects/3680707d566b43ad924b84e80b851b64` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:31.902662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3680707d566b43ad924b84e80b851b64)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:31.906461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:31.962925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6775d899bb554dbd9eade8d97b675873', 'request_id': 'req-4c50564e-3038-48c4-b845-ef5c99784739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.976227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6775d899bb554dbd9eade8d97b675873', 'request_id': 'req-4c50564e-3038-48c4-b845-ef5c99784739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.979111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3680707d566b43ad924b84e80b851b64', 'request_id': 'req-53e8be32-73b8-4ac0-bddd-491fb557edbe'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.989075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3680707d566b43ad924b84e80b851b64', 'request_id': 'req-53e8be32-73b8-4ac0-bddd-491fb557edbe'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.991596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6775d899bb554dbd9eade8d97b675873', 'request_id': 'req-4c50564e-3038-48c4-b845-ef5c99784739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:31.996545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3680707d566b43ad924b84e80b851b64', 'request_id': 'req-53e8be32-73b8-4ac0-bddd-491fb557edbe'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.022864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] The token cache is being invalidate because project 3680707d566b43ad924b84e80b851b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:32.022864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.022864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.022864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 988/2012] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:31 2022] DELETE /identity/v3/projects/3680707d566b43ad924b84e80b851b64 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:32.023382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] The token cache is being invalidate because project 6775d899bb554dbd9eade8d97b675873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:32.023382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.023382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.023382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1025/2013] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:31 2022] DELETE /identity/v3/projects/6775d899bb554dbd9eade8d97b675873 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:32.023705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:32.023705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:32.028426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:32.034892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:32.119285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8be32-73b8-4ac0-bddd-491fb557edbe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:32.122159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50564e-3038-48c4-b845-ef5c99784739 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:32.124478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:32.125978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:32.125978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:32.125978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] PATH_INFO: `/v3/users/a887d9d4f38c4dd88c945e08331c3f15` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:32.127303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a887d9d4f38c4dd88c945e08331c3f15)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:32.129339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:32.129513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:32.130423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:32.131307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:32.131307 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] PATH_INFO: `/v3/users/2069cded934642b8be4e8250fca0befe` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:32.133986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2069cded934642b8be4e8250fca0befe)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:32.134455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:32.193012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a887d9d4f38c4dd88c945e08331c3f15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.202305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a887d9d4f38c4dd88c945e08331c3f15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.204124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a887d9d4f38c4dd88c945e08331c3f15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.204285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2069cded934642b8be4e8250fca0befe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.204952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2069cded934642b8be4e8250fca0befe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.205784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2069cded934642b8be4e8250fca0befe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.215822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2069cded934642b8be4e8250fca0befe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.223775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a887d9d4f38c4dd88c945e08331c3f15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.226502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:32.231097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:32.232565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2069cded934642b8be4e8250fca0befe'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.235551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a887d9d4f38c4dd88c945e08331c3f15'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.244824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 989/2014] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:32 2022] DELETE /identity/v3/users/a887d9d4f38c4dd88c945e08331c3f15 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:32.250863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1026/2015] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:32 2022] DELETE /identity/v3/users/2069cded934642b8be4e8250fca0befe => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:32.549048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:32.550688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:32.626540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0f87-643e-49af-a019-78309f4a91ff admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:32.722066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:32.723616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:32.736994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:32.737732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:32.738019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:32.738296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] PATH_INFO: `/v3/projects/1cd526b84def4794b7268cb5575f05ab` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:32.741495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd526b84def4794b7268cb5575f05ab)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:32.743348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:32.869481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd526b84def4794b7268cb5575f05ab', 'request_id': 'req-dfee406a-08c2-42a8-adf5-b490abb60f5e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.885483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79658ebc-800b-4a6b-911a-c4ff1e0d1fe6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:32.888476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd526b84def4794b7268cb5575f05ab', 'request_id': 'req-dfee406a-08c2-42a8-adf5-b490abb60f5e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.898662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd526b84def4794b7268cb5575f05ab', 'request_id': 'req-dfee406a-08c2-42a8-adf5-b490abb60f5e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.918283 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] The token cache is being invalidate because project 1cd526b84def4794b7268cb5575f05ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:32.918615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.919279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-dfee406a-08c2-42a8-adf5-b490abb60f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:32.921056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 990/2016] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:32 2022] DELETE /identity/v3/projects/1cd526b84def4794b7268cb5575f05ab => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:32.926391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9d07-68fc-4d47-b2e3-b3a01892f29a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:32.926688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9d07-68fc-4d47-b2e3-b3a01892f29a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:32.927026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9d07-68fc-4d47-b2e3-b3a01892f29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.003320 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-08ed9d07-68fc-4d47-b2e3-b3a01892f29a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:33.008737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-08ed9d07-68fc-4d47-b2e3-b3a01892f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.013697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:33.016333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.016847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.018735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] PATH_INFO: `/v3/projects/9fa144b52443488fb36d21ed2d5dba86` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.022498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa144b52443488fb36d21ed2d5dba86)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:33.025146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 991/2017] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:33.026718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:33.032940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.033592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.034088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.058238 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:33.060705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.091709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 992/2018] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:33.093172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:33.105328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38007af5-3e36-41d4-8f8b-5c84e95bb791 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:33.142696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:33.151785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa144b52443488fb36d21ed2d5dba86', 'request_id': 'req-07b17679-6926-47db-b934-b2da69a6f73a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.174203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa144b52443488fb36d21ed2d5dba86', 'request_id': 'req-07b17679-6926-47db-b934-b2da69a6f73a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.206629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa144b52443488fb36d21ed2d5dba86', 'request_id': 'req-07b17679-6926-47db-b934-b2da69a6f73a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.221259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] The token cache is being invalidate because project 9fa144b52443488fb36d21ed2d5dba86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:33.221885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.223121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.227304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1027/2019] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:32 2022] DELETE /identity/v3/projects/9fa144b52443488fb36d21ed2d5dba86 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:33.228366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:33.229917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.246324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:33.248937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.248937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.248937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.251037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:33.251688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:33.253902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:33.278240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 993/2020] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:33.279782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:33.281930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.292117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:33.342402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b17679-6926-47db-b934-b2da69a6f73a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:33.356091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:33.356785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.357089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.357482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] PATH_INFO: `/v3/projects/09c78aa6653c423f8710d85249e8630b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.359067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c78aa6653c423f8710d85249e8630b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:33.360964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:33.383754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e810-6adf-499a-80e2-798fd7e1a9e8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:33.389518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:33.390179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.390469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.390746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.392937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.465711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c78aa6653c423f8710d85249e8630b', 'request_id': 'req-3d805d67-a160-4e4f-b1ab-6757a3e96025'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.475423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c78aa6653c423f8710d85249e8630b', 'request_id': 'req-3d805d67-a160-4e4f-b1ab-6757a3e96025'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.492794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c78aa6653c423f8710d85249e8630b', 'request_id': 'req-3d805d67-a160-4e4f-b1ab-6757a3e96025'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.509831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] The token cache is being invalidate because project 09c78aa6653c423f8710d85249e8630b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:33.510255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.510908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3d805d67-a160-4e4f-b1ab-6757a3e96025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:33.513250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1028/2021] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:33 2022] DELETE /identity/v3/projects/09c78aa6653c423f8710d85249e8630b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:33.514598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.514927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.515211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.523894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:33.525740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:33.537958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 994/2022] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:33.539598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:33.539913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:33.541156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.542104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.560806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeed449-6a06-447d-b8b6-4988e67733e7 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:33.652135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb1061-02b2-4145-9c7a-7a847a303c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:33.653851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb1061-02b2-4145-9c7a-7a847a303c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.655307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb1061-02b2-4145-9c7a-7a847a303c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.655603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb1061-02b2-4145-9c7a-7a847a303c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.656491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb1061-02b2-4145-9c7a-7a847a303c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:33.664518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb1061-02b2-4145-9c7a-7a847a303c0a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:33.689059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:33.696482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1029/2023] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:33.705527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:33.716025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.717300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 995/2024] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:33 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:33.723239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.729519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.730102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.739048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb8010-951e-4d63-8dfd-8c5679256197 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:33.795187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:33.801171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.865088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:33.885724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:33.886416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:33.886707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:33.887024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] PATH_INFO: `/v3/users/f2e88a17e1594c6fb157222753387fba` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:33.888366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e88a17e1594c6fb157222753387fba)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:33.890410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:33.973158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 996/2025] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:33.973994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:33.976273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:33.996751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a58eea-58e8-4589-9080-d16de32dbc4c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.012243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.012911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.013197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.014089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.015028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.017574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.029334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e88a17e1594c6fb157222753387fba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:34.030238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e88a17e1594c6fb157222753387fba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:34.031973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e88a17e1594c6fb157222753387fba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:34.046796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e88a17e1594c6fb157222753387fba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:34.055002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 997/2026] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:33 2022] POST /identity/v3/users => generated 289 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:34.057228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.063981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:34.081907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e88a17e1594c6fb157222753387fba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:34.101912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7212ed-6117-47ae-affc-03c4ec6484c4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.106240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1030/2027] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:33 2022] DELETE /identity/v3/users/f2e88a17e1594c6fb157222753387fba => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:34.107369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.118384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:34.120492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.121245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.121507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.121797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.122534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:34.122876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.124578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.157776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 998/2028] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:34.163149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.196487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1a55-02e4-4583-aa48-801f592fb36f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.249005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efeaefd-795a-4389-b037-f651e4d4ef00 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.258097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.258757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.259094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.259370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.260096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.261782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.294481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1031/2029] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:34.298713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.317638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98c154-8fdf-43ca-b30b-ec392dfd2425 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.325279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.325876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.326020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.326202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.326483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.326710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.327056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.327225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.327325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.328016 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.328839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.330582 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.359608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1032/2030] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:34.364313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.377563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c03157-ab99-45ca-978c-f88c1bda8a61 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.387231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc16d6-b773-4fe2-9cf4-a9d039018066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.388262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc16d6-b773-4fe2-9cf4-a9d039018066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.388615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc16d6-b773-4fe2-9cf4-a9d039018066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.388831 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 999/2031] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:34 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:34.389092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc16d6-b773-4fe2-9cf4-a9d039018066 admin admin] PATH_INFO: `/v3/projects/6ef670623f5f43ecaab1918b17eae6bf/users/e2305d5196054db887b6500cd266614e/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.393082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.448768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc16d6-b773-4fe2-9cf4-a9d039018066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ef670623f5f43ecaab1918b17eae6bf, user_id=e2305d5196054db887b6500cd266614e, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.456265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc16d6-b773-4fe2-9cf4-a9d039018066 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.482930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1033/2032] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:34 2022] PUT /identity/v3/projects/6ef670623f5f43ecaab1918b17eae6bf/users/e2305d5196054db887b6500cd266614e/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:34.495401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.495899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.498018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.498233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:34.527913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffac2dc-688b-42fc-a73e-0dfd4b6f3305 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.568246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] MFA Rules not processed for user `e2305d5196054db887b6500cd266614e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:34.569791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:34.616187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:34.665588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.666322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.666560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.666858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.667565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.669573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.688739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1034/2033] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:34.700321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.718070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe340c-456c-43d2-8934-7e184a62b364 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.724011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.724631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.724825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1000/2034] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:34 2022] POST /identity/v3/users => generated 287 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:34.725101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.725471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.726614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.727340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.729112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.743691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db177c2-56be-48f7-b151-ddf41b6838ee admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.749558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.750394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.751133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.751594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.752597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.755213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.770561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1001/2035] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:34.772150 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.785306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f88e70-627e-4462-a953-8d78f8e2b8df admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.792844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.794303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.794752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.795269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.796478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.798499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:34.807134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1035/2036] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:34 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:34.810808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.820717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1002/2037] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:34.826081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:34.838416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401850b-3ec2-4175-9f91-a6410f44e7c9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.971242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe2061-33e6-454c-9471-9cfe82d70262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.971242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2061-33e6-454c-9471-9cfe82d70262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.971242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2061-33e6-454c-9471-9cfe82d70262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.971242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2061-33e6-454c-9471-9cfe82d70262 admin admin] PATH_INFO: `/v3/projects/9a9898e09a254c2facc7b44d997feb20/users/a34a7dd760164964827529436affd8b6/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.980678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b76bc4-c2f5-4cbc-a84d-d891b7a1f76c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:34.992993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:34.993659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:34.993946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:34.994228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:34.994949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:34.996952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.010557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe2061-33e6-454c-9471-9cfe82d70262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9898e09a254c2facc7b44d997feb20, user_id=a34a7dd760164964827529436affd8b6, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.013865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe2061-33e6-454c-9471-9cfe82d70262 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.044326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1003/2038] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:34 2022] PUT /identity/v3/projects/9a9898e09a254c2facc7b44d997feb20/users/a34a7dd760164964827529436affd8b6/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:35.060405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.061040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.061040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.065383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1036/2039] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:34 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:35.066796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.093330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9397be-2f14-43d1-9cfb-7b3ed834a6f1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.131964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] MFA Rules not processed for user `a34a7dd760164964827529436affd8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:35.133322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:35.220355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:35.224944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.226211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.226557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.226883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.227622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.229999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.242450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1037/2040] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:35.247799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.262787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e6aa1-54ed-4dcb-9720-0c2d994e5fbd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.270033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.270718 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.271322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.271573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] PATH_INFO: `/v3/projects/fdc8e955cb664b5a89115238880be221/users/274de4b1bac34c8a8ff9a762f0cb6891/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.276513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1004/2041] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:35.284796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.286459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:35.301185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc8e955cb664b5a89115238880be221, user_id=274de4b1bac34c8a8ff9a762f0cb6891, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.306138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.331124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1038/2042] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:35 2022] PUT /identity/v3/projects/fdc8e955cb664b5a89115238880be221/users/274de4b1bac34c8a8ff9a762f0cb6891/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:35.335621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.347831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:35.401399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d60b-305e-4b15-93a7-f169863feebe None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.495840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.497056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.497508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.497932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.500789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7520e5-cc7c-46ad-be73-839ba4721e6b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.506647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.507454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.507729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.508061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.508793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.511719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.529355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1039/2043] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:35.530063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.541554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134c112-068f-403f-9cb5-e6cf83a63dd3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.546613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a9800-61eb-4490-bb22-b273e2d5c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.547353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a9800-61eb-4490-bb22-b273e2d5c8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.547897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a9800-61eb-4490-bb22-b273e2d5c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.548195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a9800-61eb-4490-bb22-b273e2d5c8ff admin admin] PATH_INFO: `/v3/projects/fdc8e955cb664b5a89115238880be221/users/274de4b1bac34c8a8ff9a762f0cb6891/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.551121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a9800-61eb-4490-bb22-b273e2d5c8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc8e955cb664b5a89115238880be221, user_id=274de4b1bac34c8a8ff9a762f0cb6891, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.554090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a9800-61eb-4490-bb22-b273e2d5c8ff admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.578677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1040/2044] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:35 2022] PUT /identity/v3/projects/fdc8e955cb664b5a89115238880be221/users/274de4b1bac34c8a8ff9a762f0cb6891/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:35.581599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.582001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.582292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.616621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.618552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.637159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] MFA Rules not processed for user `274de4b1bac34c8a8ff9a762f0cb6891`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:35.645007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:35.652403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1005/2045] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:35.698664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.717287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b72b26-2214-4277-bd41-f8f8e30fe4e8 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.792739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:35.807101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.807934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.808743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.808743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] PATH_INFO: `/v3/users/a34a7dd760164964827529436affd8b6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.818228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1041/2046] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:35.818454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a34a7dd760164964827529436affd8b6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:35.819593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:35.828288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.922155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a34a7dd760164964827529436affd8b6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:35.937005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612bcc7-dd6b-40a2-b0fd-f3da7dfe0195 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:35.937782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a34a7dd760164964827529436affd8b6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:35.938240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a34a7dd760164964827529436affd8b6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:35.944742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a7a39-765e-4057-a306-af787ece4bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:35.945860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a7a39-765e-4057-a306-af787ece4bbf service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:35.946379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a7a39-765e-4057-a306-af787ece4bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:35.946814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a7a39-765e-4057-a306-af787ece4bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:35.957474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a34a7dd760164964827529436affd8b6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:35.965251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a34a7dd760164964827529436affd8b6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:35.973410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1006/2047] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:35 2022] DELETE /identity/v3/users/a34a7dd760164964827529436affd8b6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:35.976916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:35.978354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:35.992767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:36.045832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a7a39-765e-4057-a306-af787ece4bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.048889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a7a39-765e-4057-a306-af787ece4bbf service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:36.055753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374220a-4e6a-4f0f-b7ce-63bef81878f5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:36.062677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:36.063370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1042/2048] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:36.063492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.063670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.063953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] PATH_INFO: `/v3/projects/21e50cc16ade4b278a0249a6e19267e7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.065215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.065605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.066080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.066637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e50cc16ade4b278a0249a6e19267e7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.069652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:36.089255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:36.090890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:36.100101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1043/2049] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:36.110583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:36.112704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:36.152618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e50cc16ade4b278a0249a6e19267e7', 'request_id': 'req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:36.179024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e50cc16ade4b278a0249a6e19267e7', 'request_id': 'req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:36.197344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e50cc16ade4b278a0249a6e19267e7', 'request_id': 'req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:36.200387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:36.226297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] The token cache is being invalidate because project 21e50cc16ade4b278a0249a6e19267e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:36.226800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:36.227369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5198ef-6c5f-4810-86e2-095b7f2017e0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:36.228844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36cbfd1b-0e89-4e49-8818-3d3ffc1fe6f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:36.237576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1007/2050] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:35 2022] DELETE /identity/v3/projects/21e50cc16ade4b278a0249a6e19267e7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:36.239404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295716e6-ea8f-4cb3-9ae2-f92ef177eece None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.242726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295716e6-ea8f-4cb3-9ae2-f92ef177eece None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.243221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295716e6-ea8f-4cb3-9ae2-f92ef177eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.286367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-295716e6-ea8f-4cb3-9ae2-f92ef177eece None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:36.291370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-295716e6-ea8f-4cb3-9ae2-f92ef177eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:36.333294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:36.334013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.334304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.334589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.340315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:36.424000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1008/2051] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:36.434334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.434707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.435023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.481080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:36.483357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:36.484450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.486225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:36.506102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1009/2052] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:36.507157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:36.517046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1044/2053] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:36.529884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe3a2-f53a-4950-b843-6455700c8090 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:36.567146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:36.568066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.568384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.568639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.571313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:36.571658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.573473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:36.604982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1010/2054] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:36.610436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:36.626030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3905-d1b8-49fa-ba4d-6a937997b011 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:36.641049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:36.659370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.659699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.659996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.660795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.672127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:36.738825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1045/2055] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:36 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:36.740454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:36.765110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b59a-b66f-40a9-9602-8b91a36e56b3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:36.910338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:36.911046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.911337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.911610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.912435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.914512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:36.946505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1011/2056] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:36 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:36.950423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:36.956339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:36.966343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27710e-a647-4e30-bfd3-24f697dd0ea9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:36.978864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:36.979532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:36.979821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:36.980109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:36.980843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:36.984107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.003501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1046/2057] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:37.005242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.017258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980fd08-178f-458c-a5b6-36e46fc62796 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.022421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.023149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.023437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.023872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.024611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.026573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.038873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1012/2058] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:37.042397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.057917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b5f43-86d7-4409-ae12-2e25cfd64f03 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.060724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.062639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.067620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.068929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.069475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.070032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] PATH_INFO: `/v3/projects/bfffdfdda3c74285a3b35d2e4c12ce2e/users/2c9e844d572a4d0ba697d58b1a4472f2/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.076562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6729f6d-3dbf-4e0e-ba29-c577fc86c300 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.087975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.088908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.089263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.089697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] PATH_INFO: `/v3/users/4f351a3b356c4f4a84a7089f5b1b2428` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.090866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfffdfdda3c74285a3b35d2e4c12ce2e, user_id=2c9e844d572a4d0ba697d58b1a4472f2, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.091207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f351a3b356c4f4a84a7089f5b1b2428)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.094582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.095896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.123801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1047/2059] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:37 2022] PUT /identity/v3/projects/bfffdfdda3c74285a3b35d2e4c12ce2e/users/2c9e844d572a4d0ba697d58b1a4472f2/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:37.125228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.128776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.157957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f351a3b356c4f4a84a7089f5b1b2428'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.160796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:37.169761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f351a3b356c4f4a84a7089f5b1b2428'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.171024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f351a3b356c4f4a84a7089f5b1b2428'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.192037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f351a3b356c4f4a84a7089f5b1b2428'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.200660 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-86639b37-3aee-40ea-acb5-f927fa188da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f351a3b356c4f4a84a7089f5b1b2428'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.204140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bbd51-3844-478c-8ca9-4d47e4d5d5b0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.207356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1013/2060] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:37 2022] DELETE /identity/v3/users/4f351a3b356c4f4a84a7089f5b1b2428 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:37.209027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4666faf-8ff2-49eb-bc85-1589ae7d9cd9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.209371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4666faf-8ff2-49eb-bc85-1589ae7d9cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.209650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4666faf-8ff2-49eb-bc85-1589ae7d9cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.212181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.212994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.213300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.213576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] PATH_INFO: `/v3/projects/9a9898e09a254c2facc7b44d997feb20` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.214863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9898e09a254c2facc7b44d997feb20)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.216897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.241718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b4666faf-8ff2-49eb-bc85-1589ae7d9cd9 None None] MFA Rules not processed for user `2c9e844d572a4d0ba697d58b1a4472f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:37.243607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b4666faf-8ff2-49eb-bc85-1589ae7d9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.280522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9898e09a254c2facc7b44d997feb20', 'request_id': 'req-43eb1189-5136-4f97-94bd-ce36423a57d3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.288189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9898e09a254c2facc7b44d997feb20', 'request_id': 'req-43eb1189-5136-4f97-94bd-ce36423a57d3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.297418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9898e09a254c2facc7b44d997feb20', 'request_id': 'req-43eb1189-5136-4f97-94bd-ce36423a57d3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.309423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] The token cache is being invalidate because project 9a9898e09a254c2facc7b44d997feb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:37.309800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.310460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-43eb1189-5136-4f97-94bd-ce36423a57d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:37.312274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1048/2061] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:37 2022] DELETE /identity/v3/projects/9a9898e09a254c2facc7b44d997feb20 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:37.313273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.313570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.313852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.329718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1014/2062] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:37.329718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.329718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.329718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.338856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:37.341271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.354282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:37.356323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.400426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:37.414451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1049/2063] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:37.415608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.417646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.431152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1015/2064] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:37.432442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.433801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.496705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54725c-0424-4faa-a4e0-1d96575f9ae6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.503441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc61c4-e3b9-4bb1-82c6-eca54653560e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.503698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.504523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.505208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.505518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.507976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.508271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a4058-ffbc-4f7b-9be4-7f74de02a725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.508970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a4058-ffbc-4f7b-9be4-7f74de02a725 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.509279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a4058-ffbc-4f7b-9be4-7f74de02a725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.509601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a4058-ffbc-4f7b-9be4-7f74de02a725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.521805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a4058-ffbc-4f7b-9be4-7f74de02a725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.523425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a4058-ffbc-4f7b-9be4-7f74de02a725 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.529328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:37.543081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1016/2065] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:37.546921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.547467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.548058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.589246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:37.590415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:37.601893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.606889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1017/2066] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:37.607144 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.609492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.620243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1050/2067] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:37.640369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9e67f-d1b5-4923-b45e-4c2ce64d77bf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.647529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.648179 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.648476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.648738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.649466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:37.649984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.651652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.684393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1018/2068] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:37.686528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.705212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb9343-60a8-4029-b3fe-a2722de736e2 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.717243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.718782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.720355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.720665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.722704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.728409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:37.797143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1051/2069] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:37 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:37.798865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:37.817309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e55b7-0dd0-424c-a482-d60b662d4dab admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:37.935552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:37.946890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:37.947191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:37.947457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:37.955033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:37.960145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.014218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1019/2070] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:37 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:38.019674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:38.032261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49731-7dd6-4afb-a1a1-8352b8bd4d5c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:38.041521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:38.042327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.042767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.043107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.044232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:38.046295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.065091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1052/2071] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:38.068175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:38.080548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63c945-30c3-4bd9-87d1-3d4bcdabe0bc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:38.088108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db388f1-af07-4d7a-a78c-c0e30e15c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:38.089308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db388f1-af07-4d7a-a78c-c0e30e15c162 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.089834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db388f1-af07-4d7a-a78c-c0e30e15c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.090154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db388f1-af07-4d7a-a78c-c0e30e15c162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.090964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db388f1-af07-4d7a-a78c-c0e30e15c162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:38.093152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db388f1-af07-4d7a-a78c-c0e30e15c162 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.116335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1020/2072] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:38.120619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:38.132972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4d08d-019d-4552-9ecc-4f526a0e6a94 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:38.147729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:38.149276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.151293 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.151755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] PATH_INFO: `/v3/projects/ee347198bfe3406e95874179b5a8c5e3/users/1beebe253739445e94dc9be413b247e4/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.223615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee347198bfe3406e95874179b5a8c5e3, user_id=1beebe253739445e94dc9be413b247e4, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:38.231347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.268170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1053/2073] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:38 2022] PUT /identity/v3/projects/ee347198bfe3406e95874179b5a8c5e3/users/1beebe253739445e94dc9be413b247e4/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:38.279284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.280079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.280565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.329990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:38.332064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:38.334383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] MFA Rules not processed for user `1beebe253739445e94dc9be413b247e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:38.345599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:38.462410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1021/2074] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:38.472246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:38.474951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:38.483568 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:38.497164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dda2a-8092-493c-b336-d246b4f72b32 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:38.512920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:38.513863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.514918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.515347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] PATH_INFO: `/v3/projects/209718226e80474bbdc5bb4104ddeff3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.517893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209718226e80474bbdc5bb4104ddeff3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:38.529290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.569513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:38.574856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc0249-380d-4933-b6c3-bdf143a8a6fb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:38.582372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:38.583511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.584196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.584663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.599492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:38.601274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.601675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209718226e80474bbdc5bb4104ddeff3', 'request_id': 'req-04433fa5-d9ba-48bc-bc81-d80a51f78631'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.611653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209718226e80474bbdc5bb4104ddeff3', 'request_id': 'req-04433fa5-d9ba-48bc-bc81-d80a51f78631'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.615654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1022/2075] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:38.624867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209718226e80474bbdc5bb4104ddeff3', 'request_id': 'req-04433fa5-d9ba-48bc-bc81-d80a51f78631'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.647128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] The token cache is being invalidate because project 209718226e80474bbdc5bb4104ddeff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:38.647872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.648136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.653977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:38.654468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1054/2076] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:38 2022] DELETE /identity/v3/projects/209718226e80474bbdc5bb4104ddeff3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:38.667968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305177c0-6343-4d2a-99e7-ceabaaba64b1 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:38.724956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:38.725620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:38.725904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:38.726182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] PATH_INFO: `/v3/users/16c95ced769b4ee7b23e0a4ac46d5c4c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:38.727396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c95ced769b4ee7b23e0a4ac46d5c4c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:38.729368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:38.778412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c95ced769b4ee7b23e0a4ac46d5c4c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.785691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c95ced769b4ee7b23e0a4ac46d5c4c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.786343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c95ced769b4ee7b23e0a4ac46d5c4c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.799112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c95ced769b4ee7b23e0a4ac46d5c4c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.806051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c95ced769b4ee7b23e0a4ac46d5c4c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:38.812362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1023/2077] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:38 2022] DELETE /identity/v3/users/16c95ced769b4ee7b23e0a4ac46d5c4c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:39.101896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:39.103526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:39.117240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04433fa5-d9ba-48bc-bc81-d80a51f78631 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:39.175306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:39.175974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:39.176282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:39.176546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] PATH_INFO: `/v3/projects/bace7afa706242e59685f6b1cbd17d6b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:39.177768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bace7afa706242e59685f6b1cbd17d6b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:39.179722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:39.237063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bace7afa706242e59685f6b1cbd17d6b', 'request_id': 'req-204a6f39-6fdb-4216-a815-34994d580136'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.246876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bace7afa706242e59685f6b1cbd17d6b', 'request_id': 'req-204a6f39-6fdb-4216-a815-34994d580136'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.259515 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bace7afa706242e59685f6b1cbd17d6b', 'request_id': 'req-204a6f39-6fdb-4216-a815-34994d580136'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.272073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] The token cache is being invalidate because project bace7afa706242e59685f6b1cbd17d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:39.272483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.273052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.274628 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1055/2078] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:39 2022] DELETE /identity/v3/projects/bace7afa706242e59685f6b1cbd17d6b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:39.278858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:39.280432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:39.293919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1e19-c016-4200-9aca-b7af23dc6178 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:39.361208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:39.361800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:39.362134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:39.362457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] PATH_INFO: `/v3/users/8d7348070722415d90e416fdd2781dbd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:39.363943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7348070722415d90e416fdd2781dbd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:39.367257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:39.389605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:39.455183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7348070722415d90e416fdd2781dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.467050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7348070722415d90e416fdd2781dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.467697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7348070722415d90e416fdd2781dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.483378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7348070722415d90e416fdd2781dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.492085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b061a3f3-3868-414d-a892-b58e6751e63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7348070722415d90e416fdd2781dbd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.498933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1024/2079] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:39 2022] DELETE /identity/v3/users/8d7348070722415d90e416fdd2781dbd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:39.784056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:39.785836 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:39.799213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a6f39-6fdb-4216-a815-34994d580136 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:39.811917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:39.860035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:39.860697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:39.860982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:39.861258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] PATH_INFO: `/v3/projects/6dec8819ba2744d199765e4e30a874ef` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:39.862491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dec8819ba2744d199765e4e30a874ef)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:39.864434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:39.912987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dec8819ba2744d199765e4e30a874ef', 'request_id': 'req-011addc6-f9de-4545-9650-313d9e4074f1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.919443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dec8819ba2744d199765e4e30a874ef', 'request_id': 'req-011addc6-f9de-4545-9650-313d9e4074f1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.929053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dec8819ba2744d199765e4e30a874ef', 'request_id': 'req-011addc6-f9de-4545-9650-313d9e4074f1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.940380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] The token cache is being invalidate because project 6dec8819ba2744d199765e4e30a874ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:39.940676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.941464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-011addc6-f9de-4545-9650-313d9e4074f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:39.943290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1056/2080] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:39 2022] DELETE /identity/v3/projects/6dec8819ba2744d199765e4e30a874ef => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:39.949633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:39.950092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:39.950204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:39.970849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:39.972026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:40.038621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1025/2081] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:40.047220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.047522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.047824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.069645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:40.070932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:40.079600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1057/2082] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:40.084128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.093889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784506f2-925b-4ff8-8f8c-3b9c427f477a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.098952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.099563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.099893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.100123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.100830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:40.101248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.102879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.115182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1026/2083] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:40.119372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.129256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8257b07-4ea7-456f-9189-5d2b2c75e0fb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.135506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.136142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.136411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.136685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.137358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.139319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.170255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1058/2084] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:40.174635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.185015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2136bd3-afdd-4969-ae06-56890dfe8518 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.256710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.257458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.257746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.258020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.258704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.260677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.284122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1027/2085] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:40.288484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.298991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411de4e-2f08-4fc3-b7c0-7a4cd021874a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.303822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.304439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.304720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.304992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.305705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.307337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.321590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1059/2086] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:40.322946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.338018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576b837-40cd-4974-b757-6e5b2feed3eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.342878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a182a9-dd8c-4963-889d-a414dfbd5b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.343512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a182a9-dd8c-4963-889d-a414dfbd5b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.343798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a182a9-dd8c-4963-889d-a414dfbd5b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.344149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a182a9-dd8c-4963-889d-a414dfbd5b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.344772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a182a9-dd8c-4963-889d-a414dfbd5b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.346398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a182a9-dd8c-4963-889d-a414dfbd5b5d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.347778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:40.359558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1028/2087] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:40.363817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.376062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0876d4-65f1-4387-ac1b-ab7b053fd13b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.382235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.382949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.383255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.383674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] PATH_INFO: `/v3/projects/77ad909a9695402ebf047759bd9bf3cc/users/cf9a7fc5b4264e8aaf230248fbf04b68/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.399701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ad909a9695402ebf047759bd9bf3cc, user_id=cf9a7fc5b4264e8aaf230248fbf04b68, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.402339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.424623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1060/2088] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:40 2022] PUT /identity/v3/projects/77ad909a9695402ebf047759bd9bf3cc/users/cf9a7fc5b4264e8aaf230248fbf04b68/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:40.436671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.436963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.437243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.467227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] MFA Rules not processed for user `cf9a7fc5b4264e8aaf230248fbf04b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:40.468486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:40.539149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1029/2089] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:40.543875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.591242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:40.616524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be138b5-e095-473a-9dea-0ac35240bcad admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.621616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.622255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.622547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.622868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.623548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.625507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.654813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1061/2090] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:40.659673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.677380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77038cc6-5d6a-4315-95dc-a91610e65c91 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.747517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.748593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.749101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.749584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.750789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.753674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.784708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1030/2091] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:40.786761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.801107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60982d-3733-4b23-b248-8a9826c48a48 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.807351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.807972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.808257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.808535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.809399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.811820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.824860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1062/2092] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:40.829277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.840369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a30395-4ad5-4e1f-853c-42658a91e450 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.846216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10192a9f-61da-4a32-bab9-ec36aa1e35a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.846864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10192a9f-61da-4a32-bab9-ec36aa1e35a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.847157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10192a9f-61da-4a32-bab9-ec36aa1e35a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.847416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10192a9f-61da-4a32-bab9-ec36aa1e35a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.848138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10192a9f-61da-4a32-bab9-ec36aa1e35a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.849774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10192a9f-61da-4a32-bab9-ec36aa1e35a7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.861939 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1031/2093] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:14:40.866503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:40.876757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ef846-b4e1-4b67-8bb9-b37a3807e213 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:40.882213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:40.882903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.883190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.883462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] PATH_INFO: `/v3/projects/f510872f8b7b4af3bbdfa809e95861a0/users/172bd7ce846f4d5ea9415fa8a0a5fb3a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.898748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f510872f8b7b4af3bbdfa809e95861a0, user_id=172bd7ce846f4d5ea9415fa8a0a5fb3a, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:40.901640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:40.916557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1063/2094] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:40 2022] PUT /identity/v3/projects/f510872f8b7b4af3bbdfa809e95861a0/users/172bd7ce846f4d5ea9415fa8a0a5fb3a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:40.927342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:40.927827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:40.927940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:40.958422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] MFA Rules not processed for user `172bd7ce846f4d5ea9415fa8a0a5fb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:40.960151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:41.016750 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:41.031236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1032/2095] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:41.033283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.101675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ecdb6-3e14-4e93-ba2f-bd1709cf00e0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.106623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.107306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.107595 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.107857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.108542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.110485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.139649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1064/2096] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:41 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:14:41.144913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.157372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac70482-8a38-4360-ac77-65b9ee5866db None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.223230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.223792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.224078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.224347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.225093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.227288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.250526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1033/2097] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:41 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:41.255564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.267447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2972830-d487-483e-9ee6-f85cff6f2210 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.272664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.273297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.273578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.273855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.274561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.276219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.287099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1065/2098] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:41.291478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.302277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e24ab-da08-47f1-89fb-ab35405716e5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.308443 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.309146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.309433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.309710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] PATH_INFO: `/v3/projects/382b9d85ceb24b1297b8c26a0782c04b/users/c2e5e5980f6648daa482ff24f062ae45/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.326164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=382b9d85ceb24b1297b8c26a0782c04b, user_id=c2e5e5980f6648daa482ff24f062ae45, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.328894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.344245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1034/2099] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:41 2022] PUT /identity/v3/projects/382b9d85ceb24b1297b8c26a0782c04b/users/c2e5e5980f6648daa482ff24f062ae45/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:41.348393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.359056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d44ed6-2092-413d-af22-6c1c7053665b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.420166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078b12-3f8b-43b5-a988-47de3bd51976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.420812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078b12-3f8b-43b5-a988-47de3bd51976 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.421102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078b12-3f8b-43b5-a988-47de3bd51976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.421404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078b12-3f8b-43b5-a988-47de3bd51976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.422117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078b12-3f8b-43b5-a988-47de3bd51976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.423747 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078b12-3f8b-43b5-a988-47de3bd51976 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.434523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1066/2100] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:41.438973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.451058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3397b7e-62f9-4053-a0c9-5041f3cc6386 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.457584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.458273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.458576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.459092 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] PATH_INFO: `/v3/projects/382b9d85ceb24b1297b8c26a0782c04b/users/c2e5e5980f6648daa482ff24f062ae45/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.461276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=382b9d85ceb24b1297b8c26a0782c04b, user_id=c2e5e5980f6648daa482ff24f062ae45, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.469876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.486281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1035/2101] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:41 2022] PUT /identity/v3/projects/382b9d85ceb24b1297b8c26a0782c04b/users/c2e5e5980f6648daa482ff24f062ae45/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:14:41.497027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.497176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.497453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.506028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:41.515186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.517384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:41.533076 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] MFA Rules not processed for user `c2e5e5980f6648daa482ff24f062ae45`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:41.534315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:41.578917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e5dcc-4224-4979-abb8-fff709d83a33 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.582816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.583487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.583832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.584126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.586129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:41.604464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1067/2102] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:41.617189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.618611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:41.631277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66ed44-c236-4213-ade0-16998456d958 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.635363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.636009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.636337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.636625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.647232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.649366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.659630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:41.660999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1068/2103] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:41.669801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.671315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f28a6c-ba2d-4184-98e4-5fc89aeb40ea service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.684409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1036/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:41.710725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:41.720712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8107-ed1f-4dac-8e0b-55daa595f2c9 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:41.724566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521ad18-8879-4bda-ace2-3c7afe53e70b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:41.725265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521ad18-8879-4bda-ace2-3c7afe53e70b service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:41.725536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521ad18-8879-4bda-ace2-3c7afe53e70b service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:41.725809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521ad18-8879-4bda-ace2-3c7afe53e70b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:41.803867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521ad18-8879-4bda-ace2-3c7afe53e70b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:41.805582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521ad18-8879-4bda-ace2-3c7afe53e70b service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:41.816586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1069/2105] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:45.908705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bca225-b59b-4c79-b121-837f3e9b0273 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:45.909243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bca225-b59b-4c79-b121-837f3e9b0273 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:45.909359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bca225-b59b-4c79-b121-837f3e9b0273 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:45.910666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1037/2106] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:14:45.924753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:45.925808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:45.926156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:45.949282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:45.950510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:46.002670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bca225-b59b-4c79-b121-837f3e9b0273 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.004622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-85bca225-b59b-4c79-b121-837f3e9b0273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:46.040973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1070/2107] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:14:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:14:46.057528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.059329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:46.072659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f1f2f-a398-4f30-81ac-293999e5a912 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.079647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.080036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.080323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.080594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.083277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bca225-b59b-4c79-b121-837f3e9b0273 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.087996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:46.089451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.090186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.090435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.090732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.090830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] PATH_INFO: `/v3/users/84b16be4f85b440a9e3afea73ffc7871` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.092279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b16be4f85b440a9e3afea73ffc7871)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.092403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.094025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.104051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1071/2108] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:14:46.152925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b16be4f85b440a9e3afea73ffc7871'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.163300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b16be4f85b440a9e3afea73ffc7871'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.163962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b16be4f85b440a9e3afea73ffc7871'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.176971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b16be4f85b440a9e3afea73ffc7871'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.183816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7acb68e-cdad-44d3-b952-d470033a4c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b16be4f85b440a9e3afea73ffc7871'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.189919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1038/2109] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:14:46 2022] DELETE /identity/v3/users/84b16be4f85b440a9e3afea73ffc7871 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:46.378852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.397819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9cfd7-cd6b-4cf0-bbec-cc4901eb7a05 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.463589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.464403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.464816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.465104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] PATH_INFO: `/v3/projects/c8b3087180c642a78010ed1382cf1868` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.467915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8b3087180c642a78010ed1382cf1868)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.470411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.539228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b3087180c642a78010ed1382cf1868', 'request_id': 'req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.549409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b3087180c642a78010ed1382cf1868', 'request_id': 'req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.560797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b3087180c642a78010ed1382cf1868', 'request_id': 'req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.576301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] The token cache is being invalidate because project c8b3087180c642a78010ed1382cf1868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:14:46.576590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.577298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a0a5ca9e-fbc1-4cf9-8290-29a7823bf4e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:14:46.579079 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1072/2110] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:14:46 2022] DELETE /identity/v3/projects/c8b3087180c642a78010ed1382cf1868 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:46.584383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.584753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.585102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.606892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:46.608056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:46.672918 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1039/2111] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:46.680864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.681156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.681432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.702495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:46.703733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:46.712903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1073/2112] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:46.717226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.727904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355492f-e25f-413d-ab9a-bd1856b6c0fb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.732953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.733601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.733905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.734288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.734922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:14:46.735234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.736897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.749665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1040/2113] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:14:46.754310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.764290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc2b9f-8ecc-40f5-85e2-638cf82d87ae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.769155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.769789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.770074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.770347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.771056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.772949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.806071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1074/2114] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:14:46 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:46.807312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.812668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:46.819415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1f017-3164-4f73-a5d2-cbe9f54425b4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.887439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.887987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.888272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.888555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.889217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.891188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.913786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1041/2115] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:14:46 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:14:46.917986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.928794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46323390-5e17-46a0-86dd-d5e1b0ff4c13 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.933699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.934337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.934603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.934886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.935604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.937213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.953874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1075/2116] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:46.958307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:46.969396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c21e2-0b64-4822-9574-9b0f74594f23 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:46.974996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbba75-9a68-45a5-93a4-583e8ae5e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:46.975611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbba75-9a68-45a5-93a4-583e8ae5e9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:46.975895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbba75-9a68-45a5-93a4-583e8ae5e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:46.976167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbba75-9a68-45a5-93a4-583e8ae5e9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:46.976872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbba75-9a68-45a5-93a4-583e8ae5e9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:46.978483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbba75-9a68-45a5-93a4-583e8ae5e9a2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:46.990633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1042/2117] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:14:46.995109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:47.006009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5baf-7119-4c86-bee0-22cc6a1bc9b2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:47.012648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:47.013349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:47.013634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:47.013905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] PATH_INFO: `/v3/projects/16713d66669a42439081860207e6f6d6/users/b9f094e1926c4cc6935762ec1eccdd98/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:47.029337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16713d66669a42439081860207e6f6d6, user_id=b9f094e1926c4cc6935762ec1eccdd98, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:47.032014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:47.048329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1076/2118] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:14:46 2022] PUT /identity/v3/projects/16713d66669a42439081860207e6f6d6/users/b9f094e1926c4cc6935762ec1eccdd98/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:14:47.059396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:47.059776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:47.059953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:47.089168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] MFA Rules not processed for user `b9f094e1926c4cc6935762ec1eccdd98`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:14:47.090534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:47.158295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1043/2119] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:14:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:14:47.171832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:14:47.174082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:14:47.201432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:47.255325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c0e50-99d8-469e-93b3-f27f4d2a5009 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:14:47.263353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53205e-46c5-47e7-95c1-81d3ecfee5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:14:47.264035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53205e-46c5-47e7-95c1-81d3ecfee5c1 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:14:47.264321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53205e-46c5-47e7-95c1-81d3ecfee5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:14:47.264597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53205e-46c5-47e7-95c1-81d3ecfee5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:14:47.273525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53205e-46c5-47e7-95c1-81d3ecfee5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:14:47.275158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53205e-46c5-47e7-95c1-81d3ecfee5c1 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:14:47.284329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1077/2120] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:14:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:14:57.154676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:14:57.280174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=24 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:04.181436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:04.183709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:04.255712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e4f16-e76e-42df-aacc-603f6c1b368a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:04.261896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:04.262672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:04.263091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:04.263561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] PATH_INFO: `/v3/users/cf9a7fc5b4264e8aaf230248fbf04b68` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:04.264972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9a7fc5b4264e8aaf230248fbf04b68)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:04.267116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:04.316718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9a7fc5b4264e8aaf230248fbf04b68'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:04.325774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9a7fc5b4264e8aaf230248fbf04b68'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:04.326492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9a7fc5b4264e8aaf230248fbf04b68'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:04.339595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9a7fc5b4264e8aaf230248fbf04b68'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:04.347199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9a7fc5b4264e8aaf230248fbf04b68'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:04.354276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1044/2121] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:04 2022] DELETE /identity/v3/users/cf9a7fc5b4264e8aaf230248fbf04b68 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:04.359452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:04.359931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:04.360035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:04.381401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:04.382663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:04.459058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1078/2122] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:04.466489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:04.468712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:04.540537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc841b-06a7-4638-95d0-350d5382d4a1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:04.544195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:04.544858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:04.545139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:04.545410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:04.555459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:04.556901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:04.566731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1045/2123] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:04.903302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:04.904840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:04.930426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935ac01-7950-4cc1-aa45-568554b56ec7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:04.935438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:04.936105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:04.936395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:04.936676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] PATH_INFO: `/v3/projects/77ad909a9695402ebf047759bd9bf3cc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:04.937889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ad909a9695402ebf047759bd9bf3cc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:04.940296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:04.993663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ad909a9695402ebf047759bd9bf3cc', 'request_id': 'req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.000301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ad909a9695402ebf047759bd9bf3cc', 'request_id': 'req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.008722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ad909a9695402ebf047759bd9bf3cc', 'request_id': 'req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.022488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] The token cache is being invalidate because project 77ad909a9695402ebf047759bd9bf3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:05.023229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.024174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.026448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1079/2124] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:04 2022] DELETE /identity/v3/projects/77ad909a9695402ebf047759bd9bf3cc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:05.030938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:05.044427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6d731-0e79-4791-b89d-c99c8590450e service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:05.067603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:05.118864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:05.119530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:05.119812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:05.120085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] PATH_INFO: `/v3/users/172bd7ce846f4d5ea9415fa8a0a5fb3a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:05.121259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172bd7ce846f4d5ea9415fa8a0a5fb3a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:05.123310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:05.173430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172bd7ce846f4d5ea9415fa8a0a5fb3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.185311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172bd7ce846f4d5ea9415fa8a0a5fb3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.186286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172bd7ce846f4d5ea9415fa8a0a5fb3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.199611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172bd7ce846f4d5ea9415fa8a0a5fb3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.206492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172bd7ce846f4d5ea9415fa8a0a5fb3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.212530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1046/2125] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:05 2022] DELETE /identity/v3/users/172bd7ce846f4d5ea9415fa8a0a5fb3a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:05.495749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:05.497243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:05.512175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2b384-9e2a-417c-9a83-83c3ad3a9101 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:05.576979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:05.577646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:05.577931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:05.578197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] PATH_INFO: `/v3/projects/f510872f8b7b4af3bbdfa809e95861a0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:05.579512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f510872f8b7b4af3bbdfa809e95861a0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:05.581475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:05.615241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:05.634033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f510872f8b7b4af3bbdfa809e95861a0', 'request_id': 'req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.641274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f510872f8b7b4af3bbdfa809e95861a0', 'request_id': 'req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.650879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f510872f8b7b4af3bbdfa809e95861a0', 'request_id': 'req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.661929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] The token cache is being invalidate because project f510872f8b7b4af3bbdfa809e95861a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:05.662202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.663057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.664658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1080/2126] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:05 2022] DELETE /identity/v3/projects/f510872f8b7b4af3bbdfa809e95861a0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:05.668738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:05.670265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:05.683814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073432d-17bc-4244-a950-44b15dabb736 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:05.745909 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:05.746488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:05.746752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:05.747055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] PATH_INFO: `/v3/users/c2e5e5980f6648daa482ff24f062ae45` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:05.748283 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e5e5980f6648daa482ff24f062ae45)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:05.750194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:05.799461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:05.802331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e5e5980f6648daa482ff24f062ae45'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.813504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e5e5980f6648daa482ff24f062ae45'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.814506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e5e5980f6648daa482ff24f062ae45'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.827116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e5e5980f6648daa482ff24f062ae45'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.834009 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8259f1ee-d891-423f-a8e2-70cac0ea6a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e5e5980f6648daa482ff24f062ae45'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:05.840154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1047/2127] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:05 2022] DELETE /identity/v3/users/c2e5e5980f6648daa482ff24f062ae45 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:06.120805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.123332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:06.149967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af5c72-71cd-4ab5-a36b-85c3e0a0f3f8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.202012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.202676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.202993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.203277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] PATH_INFO: `/v3/projects/382b9d85ceb24b1297b8c26a0782c04b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.204476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=382b9d85ceb24b1297b8c26a0782c04b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.206485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.258410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '382b9d85ceb24b1297b8c26a0782c04b', 'request_id': 'req-85df667a-d656-4e00-aa98-5f33e98fe332'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:06.264805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '382b9d85ceb24b1297b8c26a0782c04b', 'request_id': 'req-85df667a-d656-4e00-aa98-5f33e98fe332'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:06.272877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '382b9d85ceb24b1297b8c26a0782c04b', 'request_id': 'req-85df667a-d656-4e00-aa98-5f33e98fe332'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:06.283404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] The token cache is being invalidate because project 382b9d85ceb24b1297b8c26a0782c04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:06.283700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:06.284336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-85df667a-d656-4e00-aa98-5f33e98fe332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:06.285869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1081/2128] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:06 2022] DELETE /identity/v3/projects/382b9d85ceb24b1297b8c26a0782c04b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:06.292900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.293374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.293470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.313970 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:06.315272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:06.378915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1048/2129] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:06.388089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.388523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.388807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.409289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:06.410501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:06.419050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1082/2130] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:06.423601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.433627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bdc69-e15c-479a-88d7-a381253ed865 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.438882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.439511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.439819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.440098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.440806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:06.441114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.442722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.455191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1049/2131] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:06.459667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.468963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00b8fd-adad-41e9-88f6-f898aea33cdc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.473837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.474461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.474749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.475055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.475733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.477684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.507264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1083/2132] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:06 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:06.511438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.522298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cd633-33c9-4463-9765-e3e177178574 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.588420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.588971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.589290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.589566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.592600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.592600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.620337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1050/2133] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:06 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:06.627133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.644474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744257c-3502-4240-aa28-a51c01c8f2ef admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.650540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.651495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.651825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.652105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.652824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.654539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.666249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1084/2134] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:06.671023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.681228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dbb7d-ce11-4d28-b5c9-0fb2641634d4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.685980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229682-c6af-4169-a51a-097e67e167cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.686596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229682-c6af-4169-a51a-097e67e167cc admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.686907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229682-c6af-4169-a51a-097e67e167cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.687184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229682-c6af-4169-a51a-097e67e167cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.687888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229682-c6af-4169-a51a-097e67e167cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.689477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229682-c6af-4169-a51a-097e67e167cc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.699642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1051/2135] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:06.703954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.710452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:06.714622 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621da75b-5f4c-49e6-9d81-41f9593c60d9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.722709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.723428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.723717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.723998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] PATH_INFO: `/v3/projects/dcfd0d3d2054418b8e8276af1dab455f/users/bf878f8a5ae341098a6b77c46b1a077a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.741379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfd0d3d2054418b8e8276af1dab455f, user_id=bf878f8a5ae341098a6b77c46b1a077a, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.744155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.762090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1085/2136] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:06 2022] PUT /identity/v3/projects/dcfd0d3d2054418b8e8276af1dab455f/users/bf878f8a5ae341098a6b77c46b1a077a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:06.774554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71dea9-cbdb-438b-b88f-b87df38414cd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.774881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71dea9-cbdb-438b-b88f-b87df38414cd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.777704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71dea9-cbdb-438b-b88f-b87df38414cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.806867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0a71dea9-cbdb-438b-b88f-b87df38414cd None None] MFA Rules not processed for user `bf878f8a5ae341098a6b77c46b1a077a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:06.808146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0a71dea9-cbdb-438b-b88f-b87df38414cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:06.822084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:06.879062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1052/2137] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:06.891643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:06.893354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:06.959818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a9d8-c454-4e73-b0bd-39c06baaa061 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:06.963709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:06.964347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:06.964635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:06.964921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:06.974108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:06.976234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:06.986021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1086/2138] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:11.552041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c09a-b3a8-4d86-9c6d-159a77eec828 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:11.552529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c09a-b3a8-4d86-9c6d-159a77eec828 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:11.552622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c09a-b3a8-4d86-9c6d-159a77eec828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:11.579508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9135c09a-b3a8-4d86-9c6d-159a77eec828 None None] MFA Rules not processed for user `bf878f8a5ae341098a6b77c46b1a077a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:11.580951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9135c09a-b3a8-4d86-9c6d-159a77eec828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:11.588311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1053/2139] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:11.723004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:11.724817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:11.868426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26758eea-82f6-4874-8f4f-c1161e7eda6d service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:11.874776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:11.876548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:11.877078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:11.877715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] PATH_INFO: `/v3/users/b9f094e1926c4cc6935762ec1eccdd98` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:11.880234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f094e1926c4cc6935762ec1eccdd98)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:11.884349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:11.988507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f094e1926c4cc6935762ec1eccdd98'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:11.989134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f094e1926c4cc6935762ec1eccdd98'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:11.989766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f094e1926c4cc6935762ec1eccdd98'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.001177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f094e1926c4cc6935762ec1eccdd98'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.026808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f094e1926c4cc6935762ec1eccdd98'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.080034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1087/2140] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:11 2022] DELETE /identity/v3/users/b9f094e1926c4cc6935762ec1eccdd98 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:15:12.083716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f344021-6ed9-4430-910a-84608c77633d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:12.084117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f344021-6ed9-4430-910a-84608c77633d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:12.084504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f344021-6ed9-4430-910a-84608c77633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:12.126910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7f344021-6ed9-4430-910a-84608c77633d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:12.128193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7f344021-6ed9-4430-910a-84608c77633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:12.225030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1054/2141] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:12.235249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:12.236871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:12.242699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:12.301467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b2b05-14d5-448f-85a7-1412699ce60e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:12.305799 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff695ef-465b-4d54-9a2d-db622c455149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:12.306972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff695ef-465b-4d54-9a2d-db622c455149 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:12.306972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff695ef-465b-4d54-9a2d-db622c455149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:12.307605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff695ef-465b-4d54-9a2d-db622c455149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:12.317719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff695ef-465b-4d54-9a2d-db622c455149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:12.319579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff695ef-465b-4d54-9a2d-db622c455149 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:12.331153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1088/2142] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:12.731854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f344021-6ed9-4430-910a-84608c77633d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:12.733464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7f344021-6ed9-4430-910a-84608c77633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:12.751667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f344021-6ed9-4430-910a-84608c77633d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:12.769211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:12.770300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:12.771550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:12.772227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] PATH_INFO: `/v3/projects/16713d66669a42439081860207e6f6d6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:12.774809 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16713d66669a42439081860207e6f6d6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:12.778400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:12.848136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16713d66669a42439081860207e6f6d6', 'request_id': 'req-c53fc838-c893-412e-8f2a-edd5ed265b83'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.867015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16713d66669a42439081860207e6f6d6', 'request_id': 'req-c53fc838-c893-412e-8f2a-edd5ed265b83'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.885752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16713d66669a42439081860207e6f6d6', 'request_id': 'req-c53fc838-c893-412e-8f2a-edd5ed265b83'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.898996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] The token cache is being invalidate because project 16713d66669a42439081860207e6f6d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:12.899571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.900649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c53fc838-c893-412e-8f2a-edd5ed265b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:12.902300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1055/2143] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:12 2022] DELETE /identity/v3/projects/16713d66669a42439081860207e6f6d6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:12.910344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:12.910797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:12.911101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:12.931085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:12.932352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:13.004195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1089/2144] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:13.014439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.014730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.015037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.038071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:13.039825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:13.049731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1056/2145] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:13.055570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.071151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa66bab-8176-48f0-84b7-9a713fb6d403 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.077672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.078330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.078621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.078931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.079657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:13.079963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.081725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.099826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1090/2146] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:13.105005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.115946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e5688-43e0-4b25-9e5d-225c7088e1fc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.121643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.122295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.122553 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.122821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.123571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.125472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.158614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1057/2147] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:13.159984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.170620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ba81b-09bf-4f6b-b6f0-bf6a0674a75b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.229732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.230400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.230688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.231001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.231687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.233826 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.258078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1091/2148] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:13.262815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.269802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:13.274163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfada-f2a5-4fe6-9d78-0ef8510d7836 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.280059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.280886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.281356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.281818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.282555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.284255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.295260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1058/2149] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:13.299584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.310306 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04d82-38a1-4889-8d5b-8e7f130e3a0a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.315690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517001-015b-4d4b-99e4-25caf7b728af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.316441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517001-015b-4d4b-99e4-25caf7b728af admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.316736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517001-015b-4d4b-99e4-25caf7b728af admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.317011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517001-015b-4d4b-99e4-25caf7b728af admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.317733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517001-015b-4d4b-99e4-25caf7b728af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.319397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517001-015b-4d4b-99e4-25caf7b728af admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.330177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1092/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:13.334559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.345408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36243b16-7575-4845-9d74-f56756d9370a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.350558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.351287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.351574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.351881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] PATH_INFO: `/v3/projects/ed1fe71e625247c6afb3951ac4a73027/users/7e365f0d6bd4482387975d1072673243/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.367346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1fe71e625247c6afb3951ac4a73027, user_id=7e365f0d6bd4482387975d1072673243, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.370044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.384874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1059/2151] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:13 2022] PUT /identity/v3/projects/ed1fe71e625247c6afb3951ac4a73027/users/7e365f0d6bd4482387975d1072673243/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:13.395981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.396354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.396709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.419200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:13.429048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] MFA Rules not processed for user `7e365f0d6bd4482387975d1072673243`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:13.430764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:13.508807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1093/2152] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:13.513611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.582070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe48e52-0629-4d1d-bc54-74122680e873 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.587725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.588312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.588595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.588882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.589542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.591815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.620806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1060/2153] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:13.628937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.640213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4663f-f883-4bc3-b6a0-d57c1f61681e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.700412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.701146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.701400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.701703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.702369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.704332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.727316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1094/2154] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:13.731525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.742032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de5ea3-d5a6-499e-9ab8-9b895ad9c5fb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.746754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.747409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.747695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.747975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.748684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.750319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.761741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1061/2155] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:13.766063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.776350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2ee8-a05b-40f6-a514-6c00a736ff33 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.781291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.782003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.782269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.782545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] PATH_INFO: `/v3/projects/076d8b041b224dbab9d7736246313c73/users/c5c471fa452c4766b151952a3e9e4a77/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.798087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=076d8b041b224dbab9d7736246313c73, user_id=c5c471fa452c4766b151952a3e9e4a77, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.801017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.816625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1095/2156] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:13 2022] PUT /identity/v3/projects/076d8b041b224dbab9d7736246313c73/users/c5c471fa452c4766b151952a3e9e4a77/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:13.826912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.837302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e03608-0b25-4fc3-9cb0-41f5a7964023 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.894242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a8c29-2b86-4f08-9088-3273f1329d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.894872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a8c29-2b86-4f08-9088-3273f1329d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.895171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a8c29-2b86-4f08-9088-3273f1329d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.895439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a8c29-2b86-4f08-9088-3273f1329d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.896140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a8c29-2b86-4f08-9088-3273f1329d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.897762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a8c29-2b86-4f08-9088-3273f1329d84 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.907833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1062/2157] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:13.911841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:13.922779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9cd0-2453-4c5d-8d61-f49ea8464f1c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:13.927932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:13.928707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.928893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.929287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] PATH_INFO: `/v3/projects/076d8b041b224dbab9d7736246313c73/users/c5c471fa452c4766b151952a3e9e4a77/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.931406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076d8b041b224dbab9d7736246313c73, user_id=c5c471fa452c4766b151952a3e9e4a77, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:13.934034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:13.945041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1096/2158] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:13 2022] PUT /identity/v3/projects/076d8b041b224dbab9d7736246313c73/users/c5c471fa452c4766b151952a3e9e4a77/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:13.955767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:13.956051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:13.956330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:13.984531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] MFA Rules not processed for user `c5c471fa452c4766b151952a3e9e4a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:13.985754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:14.018007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:14.058667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1063/2159] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:14.072156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:14.074270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:14.136077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37d12b-0f71-4e90-a149-cedb45aae620 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:14.140406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff08420-c89b-49e8-b966-4065ebcdbbe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:14.141040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08420-c89b-49e8-b966-4065ebcdbbe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:14.141325 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08420-c89b-49e8-b966-4065ebcdbbe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:14.141599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08420-c89b-49e8-b966-4065ebcdbbe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:14.148791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:14.152332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff08420-c89b-49e8-b966-4065ebcdbbe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:14.154383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff08420-c89b-49e8-b966-4065ebcdbbe1 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:14.164527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1097/2160] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:14.232224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:14.299699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c3a94-2236-43b0-beb4-d3b96499e5d8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:14.304913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:14.305560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:14.305844 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:14.306122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] PATH_INFO: `/v3/users/7e365f0d6bd4482387975d1072673243` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:14.307386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e365f0d6bd4482387975d1072673243)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:14.309326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:14.354878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e365f0d6bd4482387975d1072673243'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:14.362448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e365f0d6bd4482387975d1072673243'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:14.363123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e365f0d6bd4482387975d1072673243'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:14.375794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e365f0d6bd4482387975d1072673243'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:14.383059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e365f0d6bd4482387975d1072673243'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:14.388926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1064/2161] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:14 2022] DELETE /identity/v3/users/7e365f0d6bd4482387975d1072673243 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:14.393596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:14.393887 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:14.394168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:14.413311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:14.414527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:14.482531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1098/2162] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:14.494948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:14.496492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:14.558384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:14.575183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd510d-0aa6-40f3-b37f-9f452368e626 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:14.579808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:14.581020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:14.581400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:14.581649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:14.592116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:14.593581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:14.604288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1065/2163] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:14.927188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:14.928890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:14.951772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1bca0-dea9-46b7-8d4a-ac9a96309c95 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:14.957512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:14.958205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:14.958495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:14.958776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] PATH_INFO: `/v3/projects/ed1fe71e625247c6afb3951ac4a73027` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:14.960181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1fe71e625247c6afb3951ac4a73027)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:14.962135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:15.057244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1fe71e625247c6afb3951ac4a73027', 'request_id': 'req-8c75e511-95bd-48be-bf01-2a82b4d258a3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.066172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1fe71e625247c6afb3951ac4a73027', 'request_id': 'req-8c75e511-95bd-48be-bf01-2a82b4d258a3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.083333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1fe71e625247c6afb3951ac4a73027', 'request_id': 'req-8c75e511-95bd-48be-bf01-2a82b4d258a3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.105978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] The token cache is being invalidate because project ed1fe71e625247c6afb3951ac4a73027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:15.106208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.106995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.109270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1099/2164] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:14 2022] DELETE /identity/v3/projects/ed1fe71e625247c6afb3951ac4a73027 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:15.113198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:15.123916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f09f7-e487-4851-a47e-bb4fe190759b service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:15.193269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:15.193944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:15.194231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:15.194558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] PATH_INFO: `/v3/users/c5c471fa452c4766b151952a3e9e4a77` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:15.195759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c471fa452c4766b151952a3e9e4a77)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:15.198294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:15.251191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c471fa452c4766b151952a3e9e4a77'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.261358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c471fa452c4766b151952a3e9e4a77'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.262265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c471fa452c4766b151952a3e9e4a77'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.276119 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c471fa452c4766b151952a3e9e4a77'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.284794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-77a14acf-9506-478d-a469-bd2398d2b81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c471fa452c4766b151952a3e9e4a77'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.292309 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1066/2165] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:15 2022] DELETE /identity/v3/users/c5c471fa452c4766b151952a3e9e4a77 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:15.634436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:15.636381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:15.672418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75e511-95bd-48be-bf01-2a82b4d258a3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:15.766763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:15.767574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:15.767898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:15.768197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] PATH_INFO: `/v3/projects/076d8b041b224dbab9d7736246313c73` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:15.770512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=076d8b041b224dbab9d7736246313c73)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:15.772850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:15.803525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:15.838203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076d8b041b224dbab9d7736246313c73', 'request_id': 'req-3ba89a55-c0be-43e6-8571-e0aecf045620'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.845501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076d8b041b224dbab9d7736246313c73', 'request_id': 'req-3ba89a55-c0be-43e6-8571-e0aecf045620'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.854715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076d8b041b224dbab9d7736246313c73', 'request_id': 'req-3ba89a55-c0be-43e6-8571-e0aecf045620'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.870776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] The token cache is being invalidate because project 076d8b041b224dbab9d7736246313c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:15.870776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.871185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ba89a55-c0be-43e6-8571-e0aecf045620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:15.873469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1100/2166] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:15 2022] DELETE /identity/v3/projects/076d8b041b224dbab9d7736246313c73 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:15.878349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:15.878637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:15.878814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:15.900897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:15.902162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:15.974652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1067/2167] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:15.983831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:15.984380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:15.984469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.008586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:16.009915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:16.024333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1101/2168] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:16.027306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.041651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f4c69-7c5c-4d9a-9997-4e296fd608a8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.048487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.049421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.049708 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.050116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.050813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:16.051201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.053364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.054880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:16.077131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1068/2169] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:16.083050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.095315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1796f-e97c-4519-8033-8bbe65e05bc5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.100680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.101987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.102186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.102510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.103254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.105625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.139253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.141977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:16.163745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16150f1f-3ae6-481a-9eb9-95b6f4f39b24 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.167815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1102/2170] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:16 2022] POST /identity/v3/projects => generated 367 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:16.171011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.271128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109f36-4972-43e4-8f08-87819dd97b9f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.278782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.279741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.280176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.280604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.281438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.284645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.285372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.286500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.287151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.288132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] PATH_INFO: `/v3/users/1beebe253739445e94dc9be413b247e4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.289670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1beebe253739445e94dc9be413b247e4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.291867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.318128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1103/2171] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:16 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:16.322512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.336226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d0734-b402-486a-ac50-a38f646f9318 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.343897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.345052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.345452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.346167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.350183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.352171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.366738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1beebe253739445e94dc9be413b247e4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:16.367244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1104/2172] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:15:16.371171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.379423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1beebe253739445e94dc9be413b247e4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:16.380563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1beebe253739445e94dc9be413b247e4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:16.385849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059f3a-196e-4272-9a37-2f12ce476b2e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.392065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240f02f-9828-4d39-a49e-cc653718189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.393406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f02f-9828-4d39-a49e-cc653718189f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.394008 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f02f-9828-4d39-a49e-cc653718189f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.394476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f02f-9828-4d39-a49e-cc653718189f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.395154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1beebe253739445e94dc9be413b247e4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:16.395685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240f02f-9828-4d39-a49e-cc653718189f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.398886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240f02f-9828-4d39-a49e-cc653718189f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.403532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1beebe253739445e94dc9be413b247e4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:16.411577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1069/2173] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:16 2022] DELETE /identity/v3/users/1beebe253739445e94dc9be413b247e4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:16.411855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1105/2174] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:15:16.416618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.417598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.418416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.418965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.420027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:16.449032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:16.449032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:16.504480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:16.517029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0de1d-2612-4f37-89b3-6c66d1231bd4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.521952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1b8cd-2343-4c69-9e9d-a65d5e1926a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.522639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b8cd-2343-4c69-9e9d-a65d5e1926a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.522951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b8cd-2343-4c69-9e9d-a65d5e1926a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.523272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1106/2175] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:16.526512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b8cd-2343-4c69-9e9d-a65d5e1926a3 admin admin] PATH_INFO: `/v3/projects/6a527ec9c3064d8b93d98f74f7b931d3/users/8bf33b177a524b7da889947c03f5dde0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.533524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.535178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:16.545129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b8cd-2343-4c69-9e9d-a65d5e1926a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a527ec9c3064d8b93d98f74f7b931d3, user_id=8bf33b177a524b7da889947c03f5dde0, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.548630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b8cd-2343-4c69-9e9d-a65d5e1926a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.551888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:16.569505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1070/2176] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:16 2022] PUT /identity/v3/projects/6a527ec9c3064d8b93d98f74f7b931d3/users/8bf33b177a524b7da889947c03f5dde0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:15:16.579991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.580438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.580778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.603507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73a41c-51b3-4b0f-8dbe-3bae5e33f4fc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.624398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] MFA Rules not processed for user `8bf33b177a524b7da889947c03f5dde0`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:16.627334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:16.683043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.683043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.683043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.683043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.730749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1071/2177] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:16.735752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.771626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.773350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.785331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1107/2178] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:15:16.815210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eb6b3-a44d-4238-a7b5-a46e47001621 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.820054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:16.820871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:16.821245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:16.821610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:16.822600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:16.824604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:16.884889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1072/2179] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:16 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:16.903707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:16.918451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18947152-6bdb-4973-9644-71b9b67b7d6b service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:16.988788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:17.003601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.005122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.005425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.006165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.007474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.016336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.046802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1108/2180] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:16 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:17.051418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.065955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c116612-b786-4592-9562-5c3186ddd94c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.082029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.082690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.083001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.083280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.084047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.085894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.110269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1073/2181] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:17.115424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.125920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13049ae8-d705-458a-ac7b-e610c1ccd666 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.131999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.132177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.132589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.133316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] PATH_INFO: `/v3/projects/b43a657be91b4ca189d02cd23ba2a227/users/067511cdea4741458ab790b17d65aac4/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.155363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43a657be91b4ca189d02cd23ba2a227, user_id=067511cdea4741458ab790b17d65aac4, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.158735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.176115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1109/2182] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:17 2022] PUT /identity/v3/projects/b43a657be91b4ca189d02cd23ba2a227/users/067511cdea4741458ab790b17d65aac4/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:17.180788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.201925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58656f72-e0ef-42ba-8a12-742db8638065 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.232975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.235300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:17.338194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.339208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.339489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.340226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.341324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.344132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.346061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:17.358643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b11ba-7d0f-4ff1-8fd3-1a2ef21566a3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.360577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1074/2183] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:17.365667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.371684 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.373601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.375189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.375674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] PATH_INFO: `/v3/projects/ee347198bfe3406e95874179b5a8c5e3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.377624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee347198bfe3406e95874179b5a8c5e3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.381342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.391300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553f6e6-55aa-423c-8680-74d0d256c8f8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.398227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fac2a8-9e8f-46e9-b04c-1009b95d8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.399705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fac2a8-9e8f-46e9-b04c-1009b95d8e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.400051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fac2a8-9e8f-46e9-b04c-1009b95d8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.400388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fac2a8-9e8f-46e9-b04c-1009b95d8e0b admin admin] PATH_INFO: `/v3/projects/b43a657be91b4ca189d02cd23ba2a227/users/067511cdea4741458ab790b17d65aac4/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.402776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fac2a8-9e8f-46e9-b04c-1009b95d8e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43a657be91b4ca189d02cd23ba2a227, user_id=067511cdea4741458ab790b17d65aac4, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.408112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fac2a8-9e8f-46e9-b04c-1009b95d8e0b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.426885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1075/2184] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:17 2022] PUT /identity/v3/projects/b43a657be91b4ca189d02cd23ba2a227/users/067511cdea4741458ab790b17d65aac4/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:17.441898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee347198bfe3406e95874179b5a8c5e3', 'request_id': 'req-f939dd2d-f042-47f0-9991-18db84b9d6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:17.443501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.443816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.444109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.450362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee347198bfe3406e95874179b5a8c5e3', 'request_id': 'req-f939dd2d-f042-47f0-9991-18db84b9d6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:17.473370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee347198bfe3406e95874179b5a8c5e3', 'request_id': 'req-f939dd2d-f042-47f0-9991-18db84b9d6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:17.501726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] The token cache is being invalidate because project ee347198bfe3406e95874179b5a8c5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:17.503156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:17.505585 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f939dd2d-f042-47f0-9991-18db84b9d6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:17.507727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] MFA Rules not processed for user `067511cdea4741458ab790b17d65aac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:17.510348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:17.532594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1110/2185] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:17 2022] DELETE /identity/v3/projects/ee347198bfe3406e95874179b5a8c5e3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:15:17.539677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddac3a-26ec-42b5-81b5-324fbb71987a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.540099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddac3a-26ec-42b5-81b5-324fbb71987a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.542540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddac3a-26ec-42b5-81b5-324fbb71987a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.613643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f3ddac3a-26ec-42b5-81b5-324fbb71987a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:17.614960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f3ddac3a-26ec-42b5-81b5-324fbb71987a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:17.677985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1076/2186] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:17.695838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.697300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:17.744721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:17.797026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1111/2187] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:17.798507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.798803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.799105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.811859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240608d-3bd1-4aa9-accc-4751be9aeded None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.821069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.821799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.822093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.822382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.822661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:17.824255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:17.834638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.836812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.836930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1112/2188] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:17.837859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.847449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e444f20-4d81-402b-a868-5a8f6536084b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.852316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1077/2189] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:15:17.853550 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.854970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.855320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.855659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.856443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:17.856928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.859273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.882330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1113/2190] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:17.883226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.901995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8024e8-41d7-4513-9115-d28d8a154a1c service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:17.909930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:17.910292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:17.910616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:17.910969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:17.912337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:17.914716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:17.974936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1078/2191] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:17 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:17.978477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:17.993125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429a1f-474a-472a-a771-9b935250e884 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:18.055722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:18.064813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:18.065604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.066156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.066345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.067148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:18.069350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:18.117370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1114/2192] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:17 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:18.121650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:18.133519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa908f-a6af-4de6-ab27-fc0403a1dce0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:18.134867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:18.138713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:18.139499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.139848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.140186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.140971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:18.143109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:18.154559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34758-57ab-4b55-8101-f354d413596e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:18.159767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:18.161033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.161502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.163044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.166710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1115/2193] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:18.167795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:18.169259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:18.194629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a5ed6-e16d-41fc-8f4c-c51f4e1e0bb1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:18.207304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:18.207962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.208242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.208512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.209224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:18.210864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:18.226401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1116/2194] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:18.227381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:18.238040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac06d3f-11f6-4fac-9251-bfc18d8df37a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:18.243231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79a870-fc46-4d4c-9e9a-c17af64a979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:18.243893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79a870-fc46-4d4c-9e9a-c17af64a979d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.244176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79a870-fc46-4d4c-9e9a-c17af64a979d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.244455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79a870-fc46-4d4c-9e9a-c17af64a979d admin admin] PATH_INFO: `/v3/projects/d14de33f76ea44db8899900481299b87/users/d9cb42db2f9a4b80b17bef0a86945a6c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.263314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79a870-fc46-4d4c-9e9a-c17af64a979d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14de33f76ea44db8899900481299b87, user_id=d9cb42db2f9a4b80b17bef0a86945a6c, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:18.267320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79a870-fc46-4d4c-9e9a-c17af64a979d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:18.305959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1117/2195] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:18 2022] PUT /identity/v3/projects/d14de33f76ea44db8899900481299b87/users/d9cb42db2f9a4b80b17bef0a86945a6c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:18.311400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.311902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.312149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.312401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:18.330482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:18.345737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:18.347170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:18.357285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1079/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:15:18.414289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1118/2197] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:18.424117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:18.425739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:18.546996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c97a7e-8e6a-4e72-be1f-e7dbd86d8c1a service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:18.655665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eb0d9-2979-4067-bcfd-c83acaea8ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:18.656349 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eb0d9-2979-4067-bcfd-c83acaea8ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:18.656638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eb0d9-2979-4067-bcfd-c83acaea8ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:18.656921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eb0d9-2979-4067-bcfd-c83acaea8ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:18.667504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eb0d9-2979-4067-bcfd-c83acaea8ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:18.668995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eb0d9-2979-4067-bcfd-c83acaea8ddc service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:18.683337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1080/2198] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:19.972457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:19.974637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:19.986929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d139541-8be3-4129-a2ba-32e7528b9d67 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:19.992603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:19.993547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:19.994067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:19.994455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] PATH_INFO: `/v3/users/e2305d5196054db887b6500cd266614e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:19.995846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2305d5196054db887b6500cd266614e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:19.998111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:20.011636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:20.068630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2305d5196054db887b6500cd266614e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.069953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2305d5196054db887b6500cd266614e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.071072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2305d5196054db887b6500cd266614e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.079957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2305d5196054db887b6500cd266614e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.098433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2305d5196054db887b6500cd266614e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.120011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1119/2199] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:19 2022] DELETE /identity/v3/users/e2305d5196054db887b6500cd266614e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:20.125237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:20.125875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:20.125990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:20.145677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:20.146936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:20.224243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1081/2200] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:20.235159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:20.236734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:20.321558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c9f43-fce5-4910-88ee-965b9ac1a876 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:20.327957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:20.329012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:20.329012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:20.329281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:20.342271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:20.344005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:20.362197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1120/2201] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:20.807031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:20.808541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:20.818669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:20.829160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b867d-7ad1-4ea6-9100-1d27d7d5f0e2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:20.843198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:20.850256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:20.851040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:20.851562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] PATH_INFO: `/v3/projects/6ef670623f5f43ecaab1918b17eae6bf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:20.853541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ef670623f5f43ecaab1918b17eae6bf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:20.855634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:20.951166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ef670623f5f43ecaab1918b17eae6bf', 'request_id': 'req-446f9313-0def-4582-8663-4602dc400b28'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.963163 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ef670623f5f43ecaab1918b17eae6bf', 'request_id': 'req-446f9313-0def-4582-8663-4602dc400b28'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.981455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ef670623f5f43ecaab1918b17eae6bf', 'request_id': 'req-446f9313-0def-4582-8663-4602dc400b28'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.998267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] The token cache is being invalidate because project 6ef670623f5f43ecaab1918b17eae6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:20.999097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:20.999546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:21.001867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1082/2202] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:20 2022] DELETE /identity/v3/projects/6ef670623f5f43ecaab1918b17eae6bf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:21.007519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:21.019991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4938fd-ef81-4375-bb5f-94d97790196b service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:21.108624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:21.110071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:21.110444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:21.111551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] PATH_INFO: `/v3/users/274de4b1bac34c8a8ff9a762f0cb6891` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:21.112290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=274de4b1bac34c8a8ff9a762f0cb6891)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:21.114288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:21.158095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:21.173173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274de4b1bac34c8a8ff9a762f0cb6891'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:21.173830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274de4b1bac34c8a8ff9a762f0cb6891'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:21.174934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274de4b1bac34c8a8ff9a762f0cb6891'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:21.182395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274de4b1bac34c8a8ff9a762f0cb6891'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:21.196982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56f33257-d5f3-448a-b267-7d8d0ecf3f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274de4b1bac34c8a8ff9a762f0cb6891'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:21.210975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1121/2203] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:21 2022] DELETE /identity/v3/users/274de4b1bac34c8a8ff9a762f0cb6891 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:22.091331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:22.094291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:22.116915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f9313-0def-4582-8663-4602dc400b28 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:22.310696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:22.344143 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:22.345097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:22.346953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] PATH_INFO: `/v3/projects/fdc8e955cb664b5a89115238880be221` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:22.351082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc8e955cb664b5a89115238880be221)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:22.363266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:22.515449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc8e955cb664b5a89115238880be221', 'request_id': 'req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:22.532067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc8e955cb664b5a89115238880be221', 'request_id': 'req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:22.539924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc8e955cb664b5a89115238880be221', 'request_id': 'req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:22.559663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] The token cache is being invalidate because project fdc8e955cb664b5a89115238880be221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:22.560022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:22.560929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8e6733fd-1ee7-4bbd-8cca-3312c8e41517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:22.562561 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1083/2204] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:22 2022] DELETE /identity/v3/projects/fdc8e955cb664b5a89115238880be221 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:22.569586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:22.570473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:22.571180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:22.597567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:22.599258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:22.694005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1122/2205] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:22.703890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:22.704257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:22.704505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:22.744084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:22.745655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:22.759284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1084/2206] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:22.762514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:22.777732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d815e7-4d84-47cb-a238-bd5207754ab3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:22.786451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:22.788495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:22.788495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:22.788884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:22.789753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:22.790227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:22.792257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:22.813319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1123/2207] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:15:22.815926 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:22.829112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab362-8f46-423e-af45-f83d62f6102b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:22.833674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:22.834323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:22.834612 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:22.834922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:22.835620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:22.837628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:22.844341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:22.874438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1085/2208] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:22 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:22.879632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:22.895433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade527-8bb7-48a4-ba93-195771c410f3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:23.025533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:23.026507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.026930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.027329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.028187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:23.030345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:23.059575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1124/2209] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:22 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:23.061638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:23.081486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ac56c-2119-4564-a02a-af49d8fda99f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:23.088311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:23.088974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.089279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.089546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.090274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:23.092049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:23.105902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1086/2210] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:23.110012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:23.121999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ae15-9942-4050-bac7-1a8cb7942bd2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:23.127783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f026b1-914e-4acc-9af4-69bc070d9cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:23.128516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f026b1-914e-4acc-9af4-69bc070d9cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.128918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f026b1-914e-4acc-9af4-69bc070d9cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.129289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f026b1-914e-4acc-9af4-69bc070d9cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.130441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f026b1-914e-4acc-9af4-69bc070d9cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:23.132149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f026b1-914e-4acc-9af4-69bc070d9cd0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:23.146380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1125/2211] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:15:23.150541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:23.161814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ac98b-5920-4861-b623-9eb744a8a820 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:23.167841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:23.168863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.169224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.169654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] PATH_INFO: `/v3/projects/a8a0f19f878844b9b0ff5e4e90201d63/users/d36ef9a0fc074d178d339b0c506875c0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.190390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a0f19f878844b9b0ff5e4e90201d63, user_id=d36ef9a0fc074d178d339b0c506875c0, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:23.193517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:23.223399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1087/2212] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:23 2022] PUT /identity/v3/projects/a8a0f19f878844b9b0ff5e4e90201d63/users/d36ef9a0fc074d178d339b0c506875c0/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:23.224541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.225089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.226262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.264837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] MFA Rules not processed for user `d36ef9a0fc074d178d339b0c506875c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:23.266110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:23.277255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:23.369371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1126/2213] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:23.406711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:23.408507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:23.524924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7eb77-a4bd-4768-9cdc-71736225f4ba admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:23.530732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9347cf1-4ab2-4b5b-841e-a44c1c956637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:23.531453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9347cf1-4ab2-4b5b-841e-a44c1c956637 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.531732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9347cf1-4ab2-4b5b-841e-a44c1c956637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.532004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9347cf1-4ab2-4b5b-841e-a44c1c956637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.556096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9347cf1-4ab2-4b5b-841e-a44c1c956637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:23.557603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9347cf1-4ab2-4b5b-841e-a44c1c956637 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:23.576428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1088/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:23.594463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:23.714583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6bc9-c890-47a7-b89c-51c27b46a494 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:23.720596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:23.721982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.722417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.722956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] PATH_INFO: `/v3/users/d36ef9a0fc074d178d339b0c506875c0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.725054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36ef9a0fc074d178d339b0c506875c0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:23.728804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:23.821349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36ef9a0fc074d178d339b0c506875c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:23.822285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36ef9a0fc074d178d339b0c506875c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:23.823102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36ef9a0fc074d178d339b0c506875c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:23.849105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36ef9a0fc074d178d339b0c506875c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:23.871885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36ef9a0fc074d178d339b0c506875c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:23.912006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1127/2215] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:23 2022] DELETE /identity/v3/users/d36ef9a0fc074d178d339b0c506875c0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:23.914175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:23.914485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:23.914746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:23.943255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:23.944893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:24.114311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1089/2216] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:24.131454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:24.134099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:24.198732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:24.274429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb7bac-3e60-45c0-a47e-259fbd2fbb1d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:24.284652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30338a1-881e-4968-ab09-418d97a630e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:24.288719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30338a1-881e-4968-ab09-418d97a630e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:24.288719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30338a1-881e-4968-ab09-418d97a630e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:24.288719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30338a1-881e-4968-ab09-418d97a630e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:24.290739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:24.292281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:24.308236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30338a1-881e-4968-ab09-418d97a630e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:24.308863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d718a-748c-48a2-b26d-9e85ecbbb4af None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:24.309716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30338a1-881e-4968-ab09-418d97a630e4 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:24.315321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:24.317335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:24.317964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:24.318568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] PATH_INFO: `/v3/users/8bf33b177a524b7da889947c03f5dde0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:24.320377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf33b177a524b7da889947c03f5dde0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:24.321598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1128/2217] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:24.323425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:24.385182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf33b177a524b7da889947c03f5dde0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:24.399774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:24.405693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf33b177a524b7da889947c03f5dde0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:24.407065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf33b177a524b7da889947c03f5dde0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:24.425725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf33b177a524b7da889947c03f5dde0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:24.434903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf33b177a524b7da889947c03f5dde0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:24.452401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1090/2218] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:24 2022] DELETE /identity/v3/users/8bf33b177a524b7da889947c03f5dde0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:24.458975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce5a4-9d0e-4bf4-8ba7-247072e92a0e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:24.459374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce5a4-9d0e-4bf4-8ba7-247072e92a0e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:24.459740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce5a4-9d0e-4bf4-8ba7-247072e92a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:24.488669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5c2ce5a4-9d0e-4bf4-8ba7-247072e92a0e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:24.490009 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5c2ce5a4-9d0e-4bf4-8ba7-247072e92a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:24.588804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1129/2219] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:24.596286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:24.598089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:24.691687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131cfc1-a9b6-4671-921c-f7d04161d240 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:24.697841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:24.698879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:24.699281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:24.699574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:24.711997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:24.713693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:24.737608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1091/2220] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:25.014964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.015537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.015537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.067125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] MFA Rules not processed for user `d9cb42db2f9a4b80b17bef0a86945a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:25.069053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.184938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1130/2221] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:25.189469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.191075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.206205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.208897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.227326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7feb53-5fb5-4e47-a075-7ee0cf2956d8 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.244239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.245577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.245860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.246132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.246850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:25.249135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce0693-fa07-40fb-845f-1eed4a473cd3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.251162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:25.261507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.263260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.263591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.264576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] PATH_INFO: `/v3/projects/a8a0f19f878844b9b0ff5e4e90201d63` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.266994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a0f19f878844b9b0ff5e4e90201d63)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:25.274528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:25.337409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1092/2222] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:25 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:25.347430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.399426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a0f19f878844b9b0ff5e4e90201d63', 'request_id': 'req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.401662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:25.412264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a0f19f878844b9b0ff5e4e90201d63', 'request_id': 'req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.429989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a0f19f878844b9b0ff5e4e90201d63', 'request_id': 'req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.444514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] The token cache is being invalidate because project a8a0f19f878844b9b0ff5e4e90201d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:25.444929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.445753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.451517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1131/2223] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:25 2022] DELETE /identity/v3/projects/a8a0f19f878844b9b0ff5e4e90201d63 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:25.452572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.454121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.456146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6a3f-8682-4eac-b6e2-6afee3c15271 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.535141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:25.537365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a367fbb-b1e6-4d0e-b248-c0b3272a3e67 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.544000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.544961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.545088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2c421-4345-4375-b714-fb26210efed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.545336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.545633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] PATH_INFO: `/v3/projects/6a527ec9c3064d8b93d98f74f7b931d3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.545787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c421-4345-4375-b714-fb26210efed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.546061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c421-4345-4375-b714-fb26210efed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.550207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c421-4345-4375-b714-fb26210efed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.550207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2c421-4345-4375-b714-fb26210efed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:25.550207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2c421-4345-4375-b714-fb26210efed3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:25.553237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a527ec9c3064d8b93d98f74f7b931d3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:25.555904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:25.581912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1093/2224] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:25 2022] POST /identity/v3/users => generated 295 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:25.583705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.584184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.584466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.607533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:25.608848 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.636647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a527ec9c3064d8b93d98f74f7b931d3', 'request_id': 'req-693b108a-3ea7-4bc8-8a16-81cb893a9b46'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.645844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a527ec9c3064d8b93d98f74f7b931d3', 'request_id': 'req-693b108a-3ea7-4bc8-8a16-81cb893a9b46'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.656553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a527ec9c3064d8b93d98f74f7b931d3', 'request_id': 'req-693b108a-3ea7-4bc8-8a16-81cb893a9b46'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.673852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] The token cache is being invalidate because project 6a527ec9c3064d8b93d98f74f7b931d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:25.674220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.674912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.676666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1132/2225] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:25 2022] DELETE /identity/v3/projects/6a527ec9c3064d8b93d98f74f7b931d3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:25.677527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.679120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.703131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1094/2226] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:25.705073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.707445 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.721999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3712-8e95-4ec7-80b6-c7b3d0a1f931 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.727430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.728260 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.728555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.728836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] PATH_INFO: `/v3/users/067511cdea4741458ab790b17d65aac4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.730100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] RBAC: Authorizing `identity:delete_user(user_id=067511cdea4741458ab790b17d65aac4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:25.732158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:25.769170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b108a-3ea7-4bc8-8a16-81cb893a9b46 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.800913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce483a33-e7a8-4b7d-b269-0b3e5dec1443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.800913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483a33-e7a8-4b7d-b269-0b3e5dec1443 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.801329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483a33-e7a8-4b7d-b269-0b3e5dec1443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.801329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce483a33-e7a8-4b7d-b269-0b3e5dec1443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.802047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce483a33-e7a8-4b7d-b269-0b3e5dec1443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:25.803675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce483a33-e7a8-4b7d-b269-0b3e5dec1443 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:25.810433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067511cdea4741458ab790b17d65aac4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.815359 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1133/2227] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:25.817140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.817510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.817821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.820865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067511cdea4741458ab790b17d65aac4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.821913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067511cdea4741458ab790b17d65aac4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.834494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:25.836219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067511cdea4741458ab790b17d65aac4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.851775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:25.853193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.855491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067511cdea4741458ab790b17d65aac4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:25.863113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1134/2228] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:25.867546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.869061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:25.873136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1095/2229] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:25 2022] DELETE /identity/v3/users/067511cdea4741458ab790b17d65aac4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:25.873136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:25.889575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133efe4e-0825-43ae-913d-92e009f2d40c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:25.966191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:25.968091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:25.968697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:25.969074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] PATH_INFO: `/v3/projects/5b33bcc707974091b1098e10867c5ccb/users/fed72656b917472aa030ad4c8ba66d2b/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:25.990080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:26.003822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b33bcc707974091b1098e10867c5ccb, user_id=fed72656b917472aa030ad4c8ba66d2b, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.004430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f578ed-e519-470d-8bad-69317b9f832f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.007065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.018305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.019099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.019388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.019666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.020384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:26.021532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.025734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.038751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1135/2230] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:25 2022] PUT /identity/v3/projects/5b33bcc707974091b1098e10867c5ccb/users/fed72656b917472aa030ad4c8ba66d2b/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:26.044786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1096/2231] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:26.047926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.049114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.062287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c989d3-5ad6-4f5b-a4f5-9e96dc8d23a3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.175464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181be69-9761-4b80-aadf-3d76cad3635c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.188153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.188808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.189101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.189381 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.190140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.191786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.197645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.199311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.199697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.200091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.200934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.203081 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.213063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1136/2232] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:26.218653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.240924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84aa21-fbc7-4420-9e63-283cc08dcf6d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.255586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1097/2233] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:26 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:26.260679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.265340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.266800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.267337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.267765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] PATH_INFO: `/v3/projects/5b33bcc707974091b1098e10867c5ccb/users/fed72656b917472aa030ad4c8ba66d2b/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.269995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b33bcc707974091b1098e10867c5ccb, user_id=fed72656b917472aa030ad4c8ba66d2b, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.273748 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.302537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1137/2234] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:26 2022] PUT /identity/v3/projects/5b33bcc707974091b1098e10867c5ccb/users/fed72656b917472aa030ad4c8ba66d2b/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:26.422368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.424957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:26.436207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2330f-21c0-4766-8e47-977f814b42da admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.561160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.562439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.562733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.563040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.563751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.565825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.606616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1098/2235] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:26 2022] POST /identity/v3/users => generated 287 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:26.611096 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfec75-437c-4266-a26a-016175ccec9c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.612928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.616229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.617033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.617415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.617789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] PATH_INFO: `/v3/projects/b43a657be91b4ca189d02cd23ba2a227` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.619897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43a657be91b4ca189d02cd23ba2a227)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.621923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.627110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ad088-2461-488e-94d9-abd5440d192a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.634432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.635062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.635202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:26.635514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.635974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.636888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.639175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.656090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1099/2236] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:26.659862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.666372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:26.686217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29d25e-3a82-4ca3-8c80-48e24fa7a046 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.694609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.695463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.695849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.696225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.697562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.700160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.716121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1100/2237] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:26.734180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:26.743418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43a657be91b4ca189d02cd23ba2a227', 'request_id': 'req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:26.747715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8fef8-0368-4cfd-ae90-3a363076dfe7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:26.752922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43a657be91b4ca189d02cd23ba2a227', 'request_id': 'req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:26.764416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43a657be91b4ca189d02cd23ba2a227', 'request_id': 'req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:26.782964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] The token cache is being invalidate because project b43a657be91b4ca189d02cd23ba2a227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:26.785258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:26.787084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f4ddeaf2-17fe-4832-b5f5-cd47b8022f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:26.794994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1138/2238] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:26 2022] DELETE /identity/v3/projects/b43a657be91b4ca189d02cd23ba2a227 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:26.820673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ce1f-1d88-48ff-a1da-0966d8e4d328 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.823414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ce1f-1d88-48ff-a1da-0966d8e4d328 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.823741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ce1f-1d88-48ff-a1da-0966d8e4d328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.839617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeda393-e00a-4da0-937a-de2aa6e9a6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:26.842549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeda393-e00a-4da0-937a-de2aa6e9a6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.842932 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeda393-e00a-4da0-937a-de2aa6e9a6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.843964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeda393-e00a-4da0-937a-de2aa6e9a6b8 admin admin] PATH_INFO: `/v3/projects/0beae1c88e9948d28a386b5e31c68f85/users/6f128878c21545e9aef8d9f9cb7b6881/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.867769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3ad9ce1f-1d88-48ff-a1da-0966d8e4d328 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:26.873555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3ad9ce1f-1d88-48ff-a1da-0966d8e4d328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:26.878719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeda393-e00a-4da0-937a-de2aa6e9a6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0beae1c88e9948d28a386b5e31c68f85, user_id=6f128878c21545e9aef8d9f9cb7b6881, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:26.882369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeda393-e00a-4da0-937a-de2aa6e9a6b8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:26.893897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1139/2239] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:26.901900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.901900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.901900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.914466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1101/2240] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:26 2022] PUT /identity/v3/projects/0beae1c88e9948d28a386b5e31c68f85/users/6f128878c21545e9aef8d9f9cb7b6881/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:15:26.920971 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:26.923029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:26.923408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:26.940462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:26.941774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:26.956307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:26.958657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:27.046637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1140/2241] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:27.051169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.061154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ced15d-2f6b-433e-91e2-d909fed513d0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.066160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.067666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.068217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.068461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.069339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:27.069621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.071536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.085683 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1141/2242] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:15:27.090936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1102/2243] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:27.094094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.105013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36523-cb50-4147-beef-0da8cba4bb73 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.110741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.112277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:27.115832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.116493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.116782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.117065 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.117759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.124362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.163609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1142/2244] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:27 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:27.173673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.201435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e56ee-f2a8-480f-8448-2bb31ae7c25e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.246243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:27.250116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:27.259498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b29d38-cfd1-4959-96a7-c49ca0fd9327 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.266631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.268632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.268632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.269815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.270572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.273035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.275375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.275965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.276192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.276468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.312734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1143/2245] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:27 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:27.318641 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.319233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.322962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.345267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c109f-b6a6-4f02-b86a-2e383fef0d6e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.350857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1103/2246] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:15:27.365239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.366703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.367133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.367518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.370555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.374416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.398272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1144/2247] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:27.402369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.415429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd831f0-51a3-49cf-8d60-0bac07feeb35 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.427383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355c9c4-b627-4fde-818a-39726641c01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.428035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355c9c4-b627-4fde-818a-39726641c01f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.428319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355c9c4-b627-4fde-818a-39726641c01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.429735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355c9c4-b627-4fde-818a-39726641c01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.430463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355c9c4-b627-4fde-818a-39726641c01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.434432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355c9c4-b627-4fde-818a-39726641c01f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.457457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1104/2248] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:27.461411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.475126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c365e-a62b-44e3-961e-5b0c20b361c6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.480574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.481289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.481602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.482064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] PATH_INFO: `/v3/projects/66ca4de700e14b0d91f1a653d6395e94/users/337fd679d9384df584ac2efaa3a4f740/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.500853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ca4de700e14b0d91f1a653d6395e94, user_id=337fd679d9384df584ac2efaa3a4f740, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.503659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:27.553392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1145/2249] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:27 2022] PUT /identity/v3/projects/66ca4de700e14b0d91f1a653d6395e94/users/337fd679d9384df584ac2efaa3a4f740/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:15:27.565007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.565534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.565902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.624814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] MFA Rules not processed for user `337fd679d9384df584ac2efaa3a4f740`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:27.627859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:27.802922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:27.804035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1105/2250] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:27.924874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b31de-5c65-43ca-a96b-400ee3a3db10 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:27.954616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:27.955370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:27.955663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:27.955997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:27.956706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:27.963635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:28.023137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:28.031495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1146/2251] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:27 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:28.037984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e363368-e57e-4ce3-ac7a-6810b32b2675 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:28.132703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:28.133360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.133652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.133930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.134789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:28.137139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:28.175011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1106/2252] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:28 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:28.180491 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:28.196395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9c206-3f3d-44db-9873-a5069808c05d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:28.207158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:28.207827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.208113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.210640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.211470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:28.216086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:28.245554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1147/2253] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:28.250257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:28.265687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960377d6-2b47-4501-969a-13503d9f0106 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:28.275653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:28.276264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.276423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.276678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] PATH_INFO: `/v3/projects/5c503d3f85d6400d9e8d59ddd9f6990a/users/4ef57b26fd844913a7cf859bbdb2195b/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.315673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c503d3f85d6400d9e8d59ddd9f6990a, user_id=4ef57b26fd844913a7cf859bbdb2195b, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:28.319593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:28.364744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:28.366390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1107/2254] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:28 2022] PUT /identity/v3/projects/5c503d3f85d6400d9e8d59ddd9f6990a/users/4ef57b26fd844913a7cf859bbdb2195b/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:28.382075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd64261-e377-4a9f-89c7-45670f8902ce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:28.521993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fb200-b1ae-40fe-9eea-4b574c1040e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:28.523020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fb200-b1ae-40fe-9eea-4b574c1040e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.523473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fb200-b1ae-40fe-9eea-4b574c1040e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.523923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fb200-b1ae-40fe-9eea-4b574c1040e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.525153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fb200-b1ae-40fe-9eea-4b574c1040e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:28.527809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fb200-b1ae-40fe-9eea-4b574c1040e9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:28.555662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:28.566890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1148/2255] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:28.569892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25525e7-391e-441a-a206-2108eef08a8f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:28.575100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:28.576053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.576639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.578397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] PATH_INFO: `/v3/projects/5c503d3f85d6400d9e8d59ddd9f6990a/users/4ef57b26fd844913a7cf859bbdb2195b/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.583847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c503d3f85d6400d9e8d59ddd9f6990a, user_id=4ef57b26fd844913a7cf859bbdb2195b, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:28.586989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:28.620120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1108/2256] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:28 2022] PUT /identity/v3/projects/5c503d3f85d6400d9e8d59ddd9f6990a/users/4ef57b26fd844913a7cf859bbdb2195b/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:28.623045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134c519-40da-49b0-887b-e00ccb7d6491 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.623510 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134c519-40da-49b0-887b-e00ccb7d6491 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.624757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134c519-40da-49b0-887b-e00ccb7d6491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.686615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7134c519-40da-49b0-887b-e00ccb7d6491 None None] MFA Rules not processed for user `4ef57b26fd844913a7cf859bbdb2195b`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:28.687907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7134c519-40da-49b0-887b-e00ccb7d6491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:28.707340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:28.869590 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1149/2257] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:28.878507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:28.884538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:28.894538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:28.894882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:28.895010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:28.895266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:28.955260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] MFA Rules not processed for user `6f128878c21545e9aef8d9f9cb7b6881`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:28.956598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:29.014498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a424510-d885-45e1-8d04-e34fc538c278 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:29.024464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:29.025330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:29.025574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:29.027275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:29.063369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:29.067725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:29.110221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1109/2258] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:29.117889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1150/2259] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:29.129187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:29.131095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:29.152627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6cdde-c1a7-4641-a7ad-9984cc3a970e service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:29.183593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478c1df-403e-4197-bd0f-521bfef0328d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:29.184256 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478c1df-403e-4197-bd0f-521bfef0328d service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:29.184543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478c1df-403e-4197-bd0f-521bfef0328d service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:29.185281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478c1df-403e-4197-bd0f-521bfef0328d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:29.206449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478c1df-403e-4197-bd0f-521bfef0328d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:29.207963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478c1df-403e-4197-bd0f-521bfef0328d service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:29.253856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1110/2260] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:30.271667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:30.701321 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c1895-069f-4db6-b69b-241d18e6dc78 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:30.707187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:30.708261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:30.708791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:30.709163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] PATH_INFO: `/v3/users/337fd679d9384df584ac2efaa3a4f740` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:30.710561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=337fd679d9384df584ac2efaa3a4f740)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:30.728708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:30.815412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:30.841445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '337fd679d9384df584ac2efaa3a4f740'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:30.844383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '337fd679d9384df584ac2efaa3a4f740'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:30.847277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '337fd679d9384df584ac2efaa3a4f740'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:30.867762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '337fd679d9384df584ac2efaa3a4f740'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:30.887199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '337fd679d9384df584ac2efaa3a4f740'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:30.925021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1151/2261] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:30 2022] DELETE /identity/v3/users/337fd679d9384df584ac2efaa3a4f740 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:30.929925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b3fd-0090-4c93-96c3-a6e80fd6b78f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:30.930534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b3fd-0090-4c93-96c3-a6e80fd6b78f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:30.930653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b3fd-0090-4c93-96c3-a6e80fd6b78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:30.963703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8ae0b3fd-0090-4c93-96c3-a6e80fd6b78f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:30.966281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8ae0b3fd-0090-4c93-96c3-a6e80fd6b78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:31.141520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1111/2262] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:31.170687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:31.177943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:31.186088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:31.187298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:31.188273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:31.255233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] MFA Rules not processed for user `6f128878c21545e9aef8d9f9cb7b6881`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:31.257433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:31.375000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aee34-9d0f-4aa5-be25-423112b1ee67 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:31.389241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:31.390489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:31.390953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:31.391442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:31.404502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:31.406315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:31.422424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1152/2263] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:31.453935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1112/2264] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:32.533406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:32.535327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:32.553759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31bae8-43b2-45fd-a4c7-263619411854 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:32.563447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:32.564309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:32.564785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:32.565592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] PATH_INFO: `/v3/projects/66ca4de700e14b0d91f1a653d6395e94` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:32.567483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ca4de700e14b0d91f1a653d6395e94)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:32.570471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:32.647126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ca4de700e14b0d91f1a653d6395e94', 'request_id': 'req-ef1e6dea-f02c-4622-976b-99229ef6d407'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:32.659792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ca4de700e14b0d91f1a653d6395e94', 'request_id': 'req-ef1e6dea-f02c-4622-976b-99229ef6d407'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:32.686932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ca4de700e14b0d91f1a653d6395e94', 'request_id': 'req-ef1e6dea-f02c-4622-976b-99229ef6d407'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:32.712533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] The token cache is being invalidate because project 66ca4de700e14b0d91f1a653d6395e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:32.712901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:32.713675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ef1e6dea-f02c-4622-976b-99229ef6d407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:32.717251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1153/2265] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:32 2022] DELETE /identity/v3/projects/66ca4de700e14b0d91f1a653d6395e94 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:32.725233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:32.744984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72796a-9d27-490a-ad4a-46c571597bf6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:32.845562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:32.879038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:32.880219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:32.880505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:32.880845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] PATH_INFO: `/v3/users/4ef57b26fd844913a7cf859bbdb2195b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:32.899985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef57b26fd844913a7cf859bbdb2195b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:32.904129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:33.025824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef57b26fd844913a7cf859bbdb2195b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:33.055934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef57b26fd844913a7cf859bbdb2195b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:33.056743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef57b26fd844913a7cf859bbdb2195b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:33.085382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef57b26fd844913a7cf859bbdb2195b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:33.107416 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef57b26fd844913a7cf859bbdb2195b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:33.116078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1113/2266] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:32 2022] DELETE /identity/v3/users/4ef57b26fd844913a7cf859bbdb2195b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:33.205754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:33.206457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:33.207737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:33.236667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:33.240217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:33.362688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:33.436775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1154/2267] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:15:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:33.506229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:33.508681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:33.658368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef7cf-3d11-4e94-b42f-f97d43afc51e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:33.668444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1e360-1218-451f-90e7-7dd76bdba80f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:33.669264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1e360-1218-451f-90e7-7dd76bdba80f service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:33.671169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1e360-1218-451f-90e7-7dd76bdba80f service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:33.671571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1e360-1218-451f-90e7-7dd76bdba80f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:33.704086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1e360-1218-451f-90e7-7dd76bdba80f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:33.709048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1e360-1218-451f-90e7-7dd76bdba80f service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:33.732497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:33.738174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:33.761439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1114/2268] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:15:33.874310 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02e799-7e4e-4c48-8161-1132556c8da0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:33.882481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:33.883211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:33.883498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:33.883774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] PATH_INFO: `/v3/projects/5c503d3f85d6400d9e8d59ddd9f6990a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:33.885668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c503d3f85d6400d9e8d59ddd9f6990a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:33.889561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:33.990193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c503d3f85d6400d9e8d59ddd9f6990a', 'request_id': 'req-6726ef01-2c7b-40e6-868e-93fc77716a27'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:34.000392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c503d3f85d6400d9e8d59ddd9f6990a', 'request_id': 'req-6726ef01-2c7b-40e6-868e-93fc77716a27'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:34.013296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c503d3f85d6400d9e8d59ddd9f6990a', 'request_id': 'req-6726ef01-2c7b-40e6-868e-93fc77716a27'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:34.028153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] The token cache is being invalidate because project 5c503d3f85d6400d9e8d59ddd9f6990a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:34.028634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:34.029702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6726ef01-2c7b-40e6-868e-93fc77716a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:34.032928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1155/2269] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:33 2022] DELETE /identity/v3/projects/5c503d3f85d6400d9e8d59ddd9f6990a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:34.039591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8da71-3382-4704-a3d1-ffb6821921ab None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.039952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8da71-3382-4704-a3d1-ffb6821921ab None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.040347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8da71-3382-4704-a3d1-ffb6821921ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.086722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-12c8da71-3382-4704-a3d1-ffb6821921ab None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:34.092710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-12c8da71-3382-4704-a3d1-ffb6821921ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:34.132246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.132565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.133638 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.226152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] MFA Rules not processed for user `fed72656b917472aa030ad4c8ba66d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:34.236200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:34.258963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:34.350278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1115/2270] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:34.358662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.358984 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.359275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.413127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:34.415628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:34.423182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:34.427751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1116/2271] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:34.436198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1156/2272] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:34.441286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:34.450528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:34.452121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:34.463475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854c75-5cb9-42e7-938f-7574e8a7d7aa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:34.483532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:34.484271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.485209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.485609 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.486486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:34.487066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:34.492720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:34.530449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1117/2273] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:34.534307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:34.557299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9487b-10bf-4a4a-a3e6-8c118c8054d1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:34.569758 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:34.570921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.571326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.571703 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.572481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:34.574501 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:34.577052 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac83ac-4da7-4db3-af3f-8cbf030a8bc2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:34.585839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:34.586719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.587130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.587507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.590652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:34.639557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1118/2274] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:34 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:34.643368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:34.666954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:34.668757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49621503-1428-4e3e-a66c-96b7b14b2bb4 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:34.679415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1157/2275] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:34.743735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1c4fb-29e3-4b46-be59-c2fcd2459808 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:34.750180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:34.751172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.751859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.752348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.753520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:34.757078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:34.774981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.775459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.775578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.822288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1119/2276] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:34 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:34.823293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:34.854205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:34.856221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:34.861689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9984079-34a0-489b-83f2-72975e88c28a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:34.873396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:34.874055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.874346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.874623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.875471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:34.881394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:34.930762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1120/2277] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:34.932581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:34.955991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc8d77-dbe1-44f0-8633-00424e2aa1ef admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:34.964306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:34.966696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:34.967177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:34.967902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:34.969425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:34.971471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:34.977023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1158/2278] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jul 28 01:15:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:35.003740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1121/2279] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:35.011970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:35.013447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:35.022429 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:35.027372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62c04-e6ea-4dbb-8948-04237448b48f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:35.046503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1b1df-77f5-4df3-8357-1218f756b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:35.049320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1b1df-77f5-4df3-8357-1218f756b25f admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:35.049712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1b1df-77f5-4df3-8357-1218f756b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:35.050058 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1b1df-77f5-4df3-8357-1218f756b25f admin admin] PATH_INFO: `/v3/projects/390c0b0512694320b72804c72157b687/users/6e8316bb035f43e2be6f6eafab9ccb40/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:35.104701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1b1df-77f5-4df3-8357-1218f756b25f admin admin] RBAC: Authorizing `identity:create_grant(project_id=390c0b0512694320b72804c72157b687, user_id=6e8316bb035f43e2be6f6eafab9ccb40, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:35.109390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1b1df-77f5-4df3-8357-1218f756b25f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:35.141137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1159/2280] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:35 2022] PUT /identity/v3/projects/390c0b0512694320b72804c72157b687/users/6e8316bb035f43e2be6f6eafab9ccb40/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:35.152942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:35.153238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:35.153627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:35.186797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e949dc-cb05-4e7b-9ecc-3b27cc055e90 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:35.207908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] MFA Rules not processed for user `6e8316bb035f43e2be6f6eafab9ccb40`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:35.209201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:35.216671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:35.220655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:35.290715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb770c-bfd6-4b16-b132-847701ad7270 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:35.291447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb770c-bfd6-4b16-b132-847701ad7270 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:35.291743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb770c-bfd6-4b16-b132-847701ad7270 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:35.292047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb770c-bfd6-4b16-b132-847701ad7270 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:35.346221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1160/2281] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:35.354216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:35.355967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:35.392528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb770c-bfd6-4b16-b132-847701ad7270 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:35.394146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb770c-bfd6-4b16-b132-847701ad7270 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:35.417940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1122/2282] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:15:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:15:35.512384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbccf9-9b90-45b6-bbbc-777f119211ee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:35.518753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:35.519861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:35.520000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:35.520227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:35.532094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:35.533999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:35.546353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1161/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:15:35.901229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282fc6a-4f06-4c9d-89f8-9eaff0ed2c87 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:35.901745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282fc6a-4f06-4c9d-89f8-9eaff0ed2c87 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:35.901849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282fc6a-4f06-4c9d-89f8-9eaff0ed2c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:35.950147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1282fc6a-4f06-4c9d-89f8-9eaff0ed2c87 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:35.952306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1282fc6a-4f06-4c9d-89f8-9eaff0ed2c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:35.990503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1123/2284] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:15:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:36.006428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:36.031811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3c6ea-8ff0-455a-ae4e-0b3a2495c834 service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:36.043679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:36.044694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:36.044902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:36.045330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:36.056954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:36.061171 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:36.092447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1162/2285] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:15:36.866617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:36.866946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:36.867206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:36.919745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:36.922698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:36.935347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1124/2286] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jul 28 01:15:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:36.954484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:36.973041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cc584-4ac7-4232-a071-70bdaaf1d1d9 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:36.983758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785d7c1-7d79-492f-ac47-dcf0e7b825aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:36.984634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d7c1-7d79-492f-ac47-dcf0e7b825aa service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:36.985095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d7c1-7d79-492f-ac47-dcf0e7b825aa service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:36.985455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d7c1-7d79-492f-ac47-dcf0e7b825aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:36.998823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785d7c1-7d79-492f-ac47-dcf0e7b825aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:37.000618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785d7c1-7d79-492f-ac47-dcf0e7b825aa service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:37.014426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1163/2287] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:15:40.223627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:40.225093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:40.345608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c228ac-3679-4602-87cb-6adddd4c1335 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:40.351446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:40.352139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:40.352641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:40.353170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] PATH_INFO: `/v3/users/bf878f8a5ae341098a6b77c46b1a077a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:40.354736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf878f8a5ae341098a6b77c46b1a077a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:40.358988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:40.452855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf878f8a5ae341098a6b77c46b1a077a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:40.457044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:40.476865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf878f8a5ae341098a6b77c46b1a077a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:40.477854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf878f8a5ae341098a6b77c46b1a077a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:40.496820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf878f8a5ae341098a6b77c46b1a077a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:40.505922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf878f8a5ae341098a6b77c46b1a077a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:40.516962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1125/2288] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:40 2022] DELETE /identity/v3/users/bf878f8a5ae341098a6b77c46b1a077a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:40.519092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:40.519925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:40.520394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:40.541240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:40.542497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:40.623352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1164/2289] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:40.634189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:40.635821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:40.727761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4b120-e072-4234-91d9-d1255cf13b84 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:40.731858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:40.733063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:40.733063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:40.733293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:40.749323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:40.751426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:40.763028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1126/2290] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:40.986127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:40.987639 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:41.006374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:41.014633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1fc4-be1c-4a26-a48b-f350d783be1c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:41.021769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:41.022640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:41.023180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:41.023467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] PATH_INFO: `/v3/projects/dcfd0d3d2054418b8e8276af1dab455f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:41.025038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcfd0d3d2054418b8e8276af1dab455f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:41.028374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:41.118101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcfd0d3d2054418b8e8276af1dab455f', 'request_id': 'req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:41.126091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcfd0d3d2054418b8e8276af1dab455f', 'request_id': 'req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:41.138422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcfd0d3d2054418b8e8276af1dab455f', 'request_id': 'req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:41.150980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] The token cache is being invalidate because project dcfd0d3d2054418b8e8276af1dab455f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:41.151278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:41.151956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e6a72d7-e400-46f7-a7ee-ee996b0343a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:41.153724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1165/2291] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:40 2022] DELETE /identity/v3/projects/dcfd0d3d2054418b8e8276af1dab455f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:43.524018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:43.526371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:43.549892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8b95e-1f75-4024-895e-1769993463bc service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:43.625203 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:43.625866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:43.626149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:43.626422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] PATH_INFO: `/v3/users/2c9e844d572a4d0ba697d58b1a4472f2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:43.627632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9e844d572a4d0ba697d58b1a4472f2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:43.629570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:43.681386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9e844d572a4d0ba697d58b1a4472f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:43.689138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9e844d572a4d0ba697d58b1a4472f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:43.689766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9e844d572a4d0ba697d58b1a4472f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:43.702727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:43.705041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9e844d572a4d0ba697d58b1a4472f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:43.718916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9e844d572a4d0ba697d58b1a4472f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:43.728167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1127/2292] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:15:43 2022] DELETE /identity/v3/users/2c9e844d572a4d0ba697d58b1a4472f2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:43.732922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:43.733561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:43.733673 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:43.764043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:43.765394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:43.829796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1166/2293] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:43.839943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:43.841578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:43.900788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf4c86-5cea-4e4f-b759-f77bd7384b9e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:43.904680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58251e-9112-45e6-8c66-4001a29c0434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:43.905319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58251e-9112-45e6-8c66-4001a29c0434 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:43.905604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58251e-9112-45e6-8c66-4001a29c0434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:43.905880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58251e-9112-45e6-8c66-4001a29c0434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:43.915100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58251e-9112-45e6-8c66-4001a29c0434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:43.916536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58251e-9112-45e6-8c66-4001a29c0434 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:43.926097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1128/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:44.272080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.274018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:44.317404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466435-9dfa-4b55-84af-cff9ab74798b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.322667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.323584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.323970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.324438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] PATH_INFO: `/v3/projects/bfffdfdda3c74285a3b35d2e4c12ce2e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.325675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfffdfdda3c74285a3b35d2e4c12ce2e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.327755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.392148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfffdfdda3c74285a3b35d2e4c12ce2e', 'request_id': 'req-afee5982-de01-47ac-a8e2-dd455c911696'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:44.407950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfffdfdda3c74285a3b35d2e4c12ce2e', 'request_id': 'req-afee5982-de01-47ac-a8e2-dd455c911696'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:44.421822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfffdfdda3c74285a3b35d2e4c12ce2e', 'request_id': 'req-afee5982-de01-47ac-a8e2-dd455c911696'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:44.436643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] The token cache is being invalidate because project bfffdfdda3c74285a3b35d2e4c12ce2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:15:44.436899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:44.437720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-afee5982-de01-47ac-a8e2-dd455c911696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:15:44.439467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1167/2295] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:15:44 2022] DELETE /identity/v3/projects/bfffdfdda3c74285a3b35d2e4c12ce2e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:44.445577 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.446473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.446473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.474095 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:44.477606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:44.545733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1129/2296] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:44.553936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.554225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.554505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.576258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:44.577571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:44.590005 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1168/2297] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:44.595072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.609683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3518d-8561-426f-bece-7337c1bbdc87 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.615651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.616293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.616583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.616860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.617578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:15:44.617892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.619546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.633560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1130/2298] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:15:44.638893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.645260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:44.649952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91223343-47bb-4b1b-9739-a50a6b398e7e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.658297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.659107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.659696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.660177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.661320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.663269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.693398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1169/2299] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:44 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:44.697134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.708005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5cbde-0533-45e1-ba4f-12ad5173410e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.769188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.769820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.770107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.770386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.771087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.773033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.796630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1131/2300] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:44 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:15:44.801154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.812789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999e0f-0b55-41c6-9dd5-130e626327e3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.818346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.819006 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.819289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.819565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.820280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.821976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.832917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1170/2301] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:44.838307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.850642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb21ee-a400-4de7-b6c5-a06f3751415d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.856159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3b644-26fc-415d-acb2-235550b8585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.856837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b644-26fc-415d-acb2-235550b8585c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.857106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b644-26fc-415d-acb2-235550b8585c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.857393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b644-26fc-415d-acb2-235550b8585c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.858122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3b644-26fc-415d-acb2-235550b8585c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.860152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3b644-26fc-415d-acb2-235550b8585c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.861705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:44.873526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1132/2302] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:44.878651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:44.889589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fd893-6cda-439f-ab4b-22d25873e3ae admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:44.894728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:44.895584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.895873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.896354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] PATH_INFO: `/v3/projects/9889594f4bf84c4591714b505f1861fe/users/70b75ae15822428f916c3403f9a67225/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.913105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9889594f4bf84c4591714b505f1861fe, user_id=70b75ae15822428f916c3403f9a67225, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:44.915913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:44.930950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1171/2303] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:44 2022] PUT /identity/v3/projects/9889594f4bf84c4591714b505f1861fe/users/70b75ae15822428f916c3403f9a67225/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:15:44.936351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae5578-18d0-4f7f-947e-ac878a3f766c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:44.936822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae5578-18d0-4f7f-947e-ac878a3f766c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:44.940083 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae5578-18d0-4f7f-947e-ac878a3f766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:44.961019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bcae5578-18d0-4f7f-947e-ac878a3f766c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:44.962259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bcae5578-18d0-4f7f-947e-ac878a3f766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:45.034356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1133/2304] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:15:45.046185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:45.048278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:45.119735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccb317-2297-4981-bd27-2e2cec51257a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:45.123616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff17d7b-e728-436a-abf7-9b9300bb07d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:45.124258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff17d7b-e728-436a-abf7-9b9300bb07d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:45.124543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff17d7b-e728-436a-abf7-9b9300bb07d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:45.124882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff17d7b-e728-436a-abf7-9b9300bb07d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:45.142640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff17d7b-e728-436a-abf7-9b9300bb07d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:45.144222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff17d7b-e728-436a-abf7-9b9300bb07d5 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:45.155492 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1172/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:15:51.419328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.419867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.432026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.439123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.440037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.441186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.466477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] MFA Rules not processed for user `70b75ae15822428f916c3403f9a67225`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:51.467866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:51.477544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] MFA Rules not processed for user `6f128878c21545e9aef8d9f9cb7b6881`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:51.479640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:15:51.553342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1173/2306] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:51.559931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1134/2307] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:15:51.564740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:51.576437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ffe81-0f83-42a5-844c-e537f2b88cb6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:51.582105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:51.582744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.583059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.583334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.584021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:51.586011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:51.620022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1174/2308] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:15:51 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:51.621943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:51.633922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474de04-020f-4ebc-8cc6-cb790a871e2f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:51.697452 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:51.698129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.698383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.698873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.699558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:51.701492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:51.727325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1135/2309] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:15:51 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:15:51.732146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:51.746854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e6c6-317c-4494-9b22-cb5d884f2e84 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:51.754232 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:51.755392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.755757 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.756017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.756875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:51.758817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:51.770706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1175/2310] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:51.775186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:51.786335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d551ee-e5f0-4fc6-9097-2b55fa863791 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:51.795995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:51.796662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.797245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.797245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] PATH_INFO: `/v3/projects/f0b12ede1da04e0d88ecb4dd72e74afa/users/35cf24f2df094b09be654f8d18b8c99a/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.816471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:51.820751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b12ede1da04e0d88ecb4dd72e74afa, user_id=35cf24f2df094b09be654f8d18b8c99a, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:51.824688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:51.841637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1136/2311] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:51 2022] PUT /identity/v3/projects/f0b12ede1da04e0d88ecb4dd72e74afa/users/35cf24f2df094b09be654f8d18b8c99a/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:51.846353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:51.859855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618518-2e87-4f98-9d8e-f1a45860328f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:51.891066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:15:51.939048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba99d0f-b955-4133-b896-2edd068015ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:51.939943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba99d0f-b955-4133-b896-2edd068015ee admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.940344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba99d0f-b955-4133-b896-2edd068015ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.940845 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba99d0f-b955-4133-b896-2edd068015ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.941764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba99d0f-b955-4133-b896-2edd068015ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:51.943619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba99d0f-b955-4133-b896-2edd068015ee admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:51.956446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1176/2312] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:15:51.960676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:15:51.973085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fd4ec-3182-4879-b088-7331a327f373 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:15:51.978912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:15:51.979579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:51.979873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:51.980212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] PATH_INFO: `/v3/projects/f0b12ede1da04e0d88ecb4dd72e74afa/users/35cf24f2df094b09be654f8d18b8c99a/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:51.982328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b12ede1da04e0d88ecb4dd72e74afa, user_id=35cf24f2df094b09be654f8d18b8c99a, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:15:51.985075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:15:51.998201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1137/2313] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:15:51 2022] PUT /identity/v3/projects/f0b12ede1da04e0d88ecb4dd72e74afa/users/35cf24f2df094b09be654f8d18b8c99a/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:15:59.801449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a75a8-4319-45a2-9b56-a382df391b30 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:15:59.803279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a75a8-4319-45a2-9b56-a382df391b30 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:15:59.803974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a75a8-4319-45a2-9b56-a382df391b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:15:59.857505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6d3a75a8-4319-45a2-9b56-a382df391b30 None None] MFA Rules not processed for user `35cf24f2df094b09be654f8d18b8c99a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:15:59.859687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6d3a75a8-4319-45a2-9b56-a382df391b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:00.008632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1177/2314] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:15:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:00.026247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:00.028432 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:00.132781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226be136-a5e4-4b2b-b850-6c1e9413a6fc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:00.139512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:00.140202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:00.140504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:00.140863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:00.228595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:00.230270 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:00.240526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1138/2315] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:16:00.396576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7587-c145-403d-bd16-ab9e7f6d2c59 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:00.396576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7587-c145-403d-bd16-ab9e7f6d2c59 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:00.396576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7587-c145-403d-bd16-ab9e7f6d2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:00.451107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-944c7587-c145-403d-bd16-ab9e7f6d2c59 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:00.452388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-944c7587-c145-403d-bd16-ab9e7f6d2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:00.535389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1178/2316] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:16:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:00.552964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:00.556480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:00.686681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8904a-6474-43a9-8c4b-f428fa34928d service glance] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:00.694392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:00.696046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:00.696587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:00.697303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:00.709244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:00.717339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:00.729606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1139/2317] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:16:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:16:01.848516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5860e7-639a-46fa-b3b2-2f1e30a027b6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:01.849138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5860e7-639a-46fa-b3b2-2f1e30a027b6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:01.849138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5860e7-639a-46fa-b3b2-2f1e30a027b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:01.871954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ea5860e7-639a-46fa-b3b2-2f1e30a027b6 None None] MFA Rules not processed for user `d9cb42db2f9a4b80b17bef0a86945a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:01.874793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ea5860e7-639a-46fa-b3b2-2f1e30a027b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:01.995420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:02.021699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1179/2318] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:02.113225 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:02.115039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:02.180923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:02.280226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c597434-1328-4396-9b4e-9f5660e4d758 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:02.288606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:02.291460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:02.291768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:02.292214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] PATH_INFO: `/v3/users/6e8316bb035f43e2be6f6eafab9ccb40` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:02.296348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8316bb035f43e2be6f6eafab9ccb40)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:02.302064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:02.435007 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8316bb035f43e2be6f6eafab9ccb40'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:02.451246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8316bb035f43e2be6f6eafab9ccb40'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:02.453514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8316bb035f43e2be6f6eafab9ccb40'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:02.493017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8316bb035f43e2be6f6eafab9ccb40'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:02.520978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8316bb035f43e2be6f6eafab9ccb40'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:02.539477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1140/2319] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:16:02 2022] DELETE /identity/v3/users/6e8316bb035f43e2be6f6eafab9ccb40 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:02.541860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:02.542158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:02.542474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:02.628014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:02.637803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:02.810001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1180/2320] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:02.821009 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:02.822713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:02.955597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41b414-3fa4-46b5-9ad9-ec291b652fd7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:02.959940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8a451-b445-40d4-9325-48b4b0aa34d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:02.960832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8a451-b445-40d4-9325-48b4b0aa34d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:02.961235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8a451-b445-40d4-9325-48b4b0aa34d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:02.961613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8a451-b445-40d4-9325-48b4b0aa34d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:02.974466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8a451-b445-40d4-9325-48b4b0aa34d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:02.976094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8a451-b445-40d4-9325-48b4b0aa34d1 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:03.027378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1141/2321] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:03.769832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:03.771506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:03.797358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2ba94-d0ad-4224-8f5d-de8a60529384 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:03.850610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:03.851390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:03.851782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:03.851948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] PATH_INFO: `/v3/projects/390c0b0512694320b72804c72157b687` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:03.853657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] RBAC: Authorizing `identity:delete_project(project_id=390c0b0512694320b72804c72157b687)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:03.860184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:03.979211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390c0b0512694320b72804c72157b687', 'request_id': 'req-eada9e14-fc8e-4641-b3de-333d55c7782e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:03.990597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390c0b0512694320b72804c72157b687', 'request_id': 'req-eada9e14-fc8e-4641-b3de-333d55c7782e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:04.012891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390c0b0512694320b72804c72157b687', 'request_id': 'req-eada9e14-fc8e-4641-b3de-333d55c7782e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:04.032092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] The token cache is being invalidate because project 390c0b0512694320b72804c72157b687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:16:04.032570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:04.033109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eada9e14-fc8e-4641-b3de-333d55c7782e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:04.034693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1181/2322] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:16:03 2022] DELETE /identity/v3/projects/390c0b0512694320b72804c72157b687 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:04.040542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:04.041508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:04.042067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:04.083628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:04.085815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:04.194450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:04.263334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1142/2323] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:04.288823 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:04.289230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:04.289541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:04.351552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:04.352816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:04.376288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1182/2324] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:04.376618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:04.414756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a7a08-aff0-4b91-9f72-ced172f39ab3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:04.433719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:04.434492 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:04.434999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:04.435384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:04.436275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:16:04.436705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:04.439780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:04.480038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1143/2325] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:16:04.482665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:04.510957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397cb34-a2bc-48d3-bb8f-327ad46db2ad None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:04.555829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:04.556759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:04.557034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:04.557445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:04.558222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:04.560340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:04.638985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1183/2326] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:16:04 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:16:04.640395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:04.670342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd03278-30e3-42fe-9eff-1cc351d00acb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:04.853981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:04.854799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:04.855218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:04.855552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:04.857129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:04.859281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:04.906564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1144/2327] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:16:04 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:16:04.910804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:04.928333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98df94-2c4b-4e30-8046-4034a86ac1a4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:04.949899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:04.950762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:04.951177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:04.951549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:04.952609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:04.955032 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:05.011329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1184/2328] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:16:05.014534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:05.040366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2d5bf-3a62-4f18-b357-2811e5d3a6c7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:05.054243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5d3d0-0628-4b7b-9bb8-b0e880b1fea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:05.058929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d3d0-0628-4b7b-9bb8-b0e880b1fea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:05.059537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d3d0-0628-4b7b-9bb8-b0e880b1fea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:05.061621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d3d0-0628-4b7b-9bb8-b0e880b1fea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:05.064279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5d3d0-0628-4b7b-9bb8-b0e880b1fea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:05.068084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5d3d0-0628-4b7b-9bb8-b0e880b1fea4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:05.125072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1145/2329] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:16:05.129557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:05.149988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e61b4-cc34-42ab-8510-b1db1b7de16d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:05.180522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:05.181364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:05.182104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:05.182104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] PATH_INFO: `/v3/projects/48737a6e4fd34fd6bfe725a79a6f8451/users/bfdce7dac0a6408094a0c9bcfb9422ea/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:05.219106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48737a6e4fd34fd6bfe725a79a6f8451, user_id=bfdce7dac0a6408094a0c9bcfb9422ea, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:05.233407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:05.304149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1185/2330] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:16:05 2022] PUT /identity/v3/projects/48737a6e4fd34fd6bfe725a79a6f8451/users/bfdce7dac0a6408094a0c9bcfb9422ea/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:16:05.317246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255bdc9-6b09-436f-a8e4-2326da44f1b7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:05.317816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255bdc9-6b09-436f-a8e4-2326da44f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:05.319534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255bdc9-6b09-436f-a8e4-2326da44f1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:05.390816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7255bdc9-6b09-436f-a8e4-2326da44f1b7 None None] MFA Rules not processed for user `bfdce7dac0a6408094a0c9bcfb9422ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:05.392542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7255bdc9-6b09-436f-a8e4-2326da44f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:05.506535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1146/2331] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:05.514456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:05.516441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:05.568713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:05.681317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df3489-19ed-4eba-9a54-57a4b80b701d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:05.688412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:05.689279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:05.689603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:05.690015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:05.705798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:05.707010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:05.719951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1186/2332] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:08.583896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f4e03-1632-4b92-b207-b93d7814ae66 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:08.584491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f4e03-1632-4b92-b207-b93d7814ae66 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:08.584491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f4e03-1632-4b92-b207-b93d7814ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:08.637384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-669f4e03-1632-4b92-b207-b93d7814ae66 None None] MFA Rules not processed for user `018eb80709e94d5e8653fb6eeb342c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:08.638580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-669f4e03-1632-4b92-b207-b93d7814ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:08.700653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:08.753268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1147/2333] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:14.387343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:14.516108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695748-3706-435a-b1da-aea41d612e86 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:14.523918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:14.524787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:14.525083 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:14.525367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] PATH_INFO: `/v3/users/bfdce7dac0a6408094a0c9bcfb9422ea` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:14.527388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfdce7dac0a6408094a0c9bcfb9422ea)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:14.529910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:14.605848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfdce7dac0a6408094a0c9bcfb9422ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:14.606435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfdce7dac0a6408094a0c9bcfb9422ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:14.607104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfdce7dac0a6408094a0c9bcfb9422ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:14.623918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfdce7dac0a6408094a0c9bcfb9422ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:14.656765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfdce7dac0a6408094a0c9bcfb9422ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:14.678697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1187/2334] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:16:14 2022] DELETE /identity/v3/users/bfdce7dac0a6408094a0c9bcfb9422ea => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:14.684286 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4f703-50ae-4771-9411-f7ac2cc9331a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:14.684863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4f703-50ae-4771-9411-f7ac2cc9331a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:14.685051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4f703-50ae-4771-9411-f7ac2cc9331a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:14.707137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c2b4f703-50ae-4771-9411-f7ac2cc9331a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:14.708557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2b4f703-50ae-4771-9411-f7ac2cc9331a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:14.808487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1148/2335] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:14.819824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:14.822104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:14.938784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:15.004121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267cb5-05c1-49b1-bfae-399d149094a8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:15.011989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:15.012830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:15.013260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:15.013675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:15.029989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:15.032249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:15.044856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1188/2336] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:15.821027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:15.822068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:15.822551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:15.858672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:15.860964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:16.011718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1149/2337] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:16:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:16.036208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:16.037469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:16.048344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:16.050911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:16.064343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441cbe-9ff0-417e-8381-fb4a9a1b235f service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:16.075949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:16.076710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.076910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.077201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] PATH_INFO: `/v3/projects/48737a6e4fd34fd6bfe725a79a6f8451` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.081526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48737a6e4fd34fd6bfe725a79a6f8451)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:16.104430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:16.195376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1ba07-21fe-439f-b278-7bb56535b108 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:16.210990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662d572-3279-41c5-8d1c-6b964278c5e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:16.213414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d572-3279-41c5-8d1c-6b964278c5e9 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.213957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d572-3279-41c5-8d1c-6b964278c5e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.214963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d572-3279-41c5-8d1c-6b964278c5e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.239645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662d572-3279-41c5-8d1c-6b964278c5e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:16.242150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48737a6e4fd34fd6bfe725a79a6f8451', 'request_id': 'req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:16.244357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662d572-3279-41c5-8d1c-6b964278c5e9 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:16.251623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48737a6e4fd34fd6bfe725a79a6f8451', 'request_id': 'req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:16.262989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1150/2338] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:16:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 01:16:16.272972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48737a6e4fd34fd6bfe725a79a6f8451', 'request_id': 'req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:16.291543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] The token cache is being invalidate because project 48737a6e4fd34fd6bfe725a79a6f8451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:16:16.291965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:16.292607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cfa6cd43-4fd3-47ce-a521-2b3c5fd7c0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:16.294778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1189/2339] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:16:16 2022] DELETE /identity/v3/projects/48737a6e4fd34fd6bfe725a79a6f8451 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:16.299751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ee9c-c878-42b3-8835-e4ff9382a8b1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.300159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ee9c-c878-42b3-8835-e4ff9382a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.300288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ee9c-c878-42b3-8835-e4ff9382a8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.343168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.343755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.344211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.392020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0802ee9c-c878-42b3-8835-e4ff9382a8b1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:16.403299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0802ee9c-c878-42b3-8835-e4ff9382a8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:16.427832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] MFA Rules not processed for user `70b75ae15822428f916c3403f9a67225`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:16.429615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:16.455301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:16.562392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1151/2340] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:16.570292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.570645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.570967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.599116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1190/2341] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:16.624736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:16.627885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:16.651990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:16.660658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1152/2342] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:16:16.667289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3766590-6689-42f9-a66d-5dc711161d60 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:16.673444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:16.674989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.675440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.675792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.676977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:16:16.677402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:16.679508 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:16.714737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1191/2343] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:16:16.714951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:16.727063 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d86c8-f0f7-4498-8fdf-78ced523635c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:16.734231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:16.734231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.734231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.734665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.735622 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:16.737284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:16.820141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:16.826285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1153/2344] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:16:16 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:16:16.832459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020ac7-5b2c-4693-9a9e-008ad0ac8e4f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:16.904991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:16.931947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:16.932876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.933608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.934048 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.934982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:16.937087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:16.967429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1192/2345] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:16:16 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:16:16.971613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:16.983726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac783b-d3b9-4c27-8550-ebcb833f949f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:16.992410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:16.993502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:16.994045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:16.994541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:16.995533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:16.997383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:17.014196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1154/2346] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:16:17.014959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:17.048980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff70c3-c2c8-48ff-8b0b-b9c543392894 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:17.065260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d647b09-524b-4702-ad85-a7acf3c9b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:17.066172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d647b09-524b-4702-ad85-a7acf3c9b7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:17.066602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d647b09-524b-4702-ad85-a7acf3c9b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:17.067078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d647b09-524b-4702-ad85-a7acf3c9b7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:17.068434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d647b09-524b-4702-ad85-a7acf3c9b7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:17.083761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d647b09-524b-4702-ad85-a7acf3c9b7d5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:17.097870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1193/2347] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:16:17.103837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:17.118421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633a867-2b29-4dc3-8f28-34078b051b94 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:17.125044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:17.126158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:17.126688 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:17.127542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] PATH_INFO: `/v3/projects/39be0436e2734bcd833b571fc15e4b50/users/cb76361c85ec437181dc077971c0d3d6/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:17.160705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39be0436e2734bcd833b571fc15e4b50, user_id=cb76361c85ec437181dc077971c0d3d6, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:17.164182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:17.203864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1155/2348] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:16:17 2022] PUT /identity/v3/projects/39be0436e2734bcd833b571fc15e4b50/users/cb76361c85ec437181dc077971c0d3d6/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:17.205558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1122-f41e-4e0e-abe8-da32345aa3c6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:17.205829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1122-f41e-4e0e-abe8-da32345aa3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:17.206138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1122-f41e-4e0e-abe8-da32345aa3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:17.275509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ffdb1122-f41e-4e0e-abe8-da32345aa3c6 None None] MFA Rules not processed for user `cb76361c85ec437181dc077971c0d3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:17.277101 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ffdb1122-f41e-4e0e-abe8-da32345aa3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:17.394861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1194/2349] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:16:17.402722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:17.404570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:17.508221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521082-dba3-452b-8604-b88f13648223 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:17.512336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:17.513295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:17.513753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:17.514262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:17.525082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:17.527259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:17.539852 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1156/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:24.426999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036a96-a5bf-4822-bda3-7a592b47c6ad None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:24.427978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036a96-a5bf-4822-bda3-7a592b47c6ad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:24.428275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036a96-a5bf-4822-bda3-7a592b47c6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:24.461690 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-94036a96-a5bf-4822-bda3-7a592b47c6ad None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:24.463553 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-94036a96-a5bf-4822-bda3-7a592b47c6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:24.565242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1195/2351] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:16:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:16:24.579963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:24.581578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:24.739864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:24.760323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418f1d6-5303-4344-98dc-f1df797c03a9 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:24.783051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:24.783900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:24.784268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:24.784619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:24.803709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:24.805653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:24.831269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1157/2352] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:16:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:16:29.499441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:29.500327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:29.500699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:29.568694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:29.570114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:29.585832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1196/2353] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:16:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:29.617546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:29.633088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6aa1-54ae-493f-a065-08dc57a2befd service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:29.646597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:29.647297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:29.647619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:29.647873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:29.663520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:29.666465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:29.707588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1158/2354] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:16:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:16:34.101181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:34.101181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:34.242570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:34.250389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:34.268944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b30358-301d-44e4-936a-c292efaeed22 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:34.285810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:34.286616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:34.287190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:34.288365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:34.294068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:34.319899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:34.321503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62176fff-7cd8-4a0e-8c1d-337ca4778494 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:34.373484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1197/2355] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:34.395529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e1ad-4557-478b-b052-c87a9d8ef148 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:34.421316 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:34.422201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:34.422491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:34.423073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] PATH_INFO: `/v3/users/cb76361c85ec437181dc077971c0d3d6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:34.425328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb76361c85ec437181dc077971c0d3d6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:34.427754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:34.625603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb76361c85ec437181dc077971c0d3d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:34.645412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb76361c85ec437181dc077971c0d3d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:34.646136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb76361c85ec437181dc077971c0d3d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:34.667858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb76361c85ec437181dc077971c0d3d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:34.677976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb76361c85ec437181dc077971c0d3d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:34.691620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1159/2356] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:16:34 2022] DELETE /identity/v3/users/cb76361c85ec437181dc077971c0d3d6 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:16:34.693408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:34.693703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:34.693984 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:34.718327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:34.719830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:34.845179 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1198/2357] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:34.860734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:34.862385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:35.031635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f20dc-2bba-4c02-a2e1-6721c0efb8a9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:35.058872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cff95-63a4-4582-82da-660dd4cde424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:35.060008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cff95-63a4-4582-82da-660dd4cde424 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:35.060301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cff95-63a4-4582-82da-660dd4cde424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:35.060583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cff95-63a4-4582-82da-660dd4cde424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:35.091343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cff95-63a4-4582-82da-660dd4cde424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:35.093603 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cff95-63a4-4582-82da-660dd4cde424 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:35.100129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:35.139751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1160/2358] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:35.538111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:35.539828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:35.559654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436847f-5ca0-4569-bfac-9228d67905ae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:35.574456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:35.576189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:35.576721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:35.577842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] PATH_INFO: `/v3/projects/39be0436e2734bcd833b571fc15e4b50` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:35.579353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39be0436e2734bcd833b571fc15e4b50)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:35.593694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:35.658244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39be0436e2734bcd833b571fc15e4b50', 'request_id': 'req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:35.667774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39be0436e2734bcd833b571fc15e4b50', 'request_id': 'req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:35.678324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39be0436e2734bcd833b571fc15e4b50', 'request_id': 'req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:35.692035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] The token cache is being invalidate because project 39be0436e2734bcd833b571fc15e4b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:16:35.692573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:35.693302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a454dd06-2f4f-48ef-a1b5-e1e6958a5530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:35.695117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1199/2359] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:16:35 2022] DELETE /identity/v3/projects/39be0436e2734bcd833b571fc15e4b50 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:44.666167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a26e7-c614-41ac-b9c6-c58f29af6af9 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:44.666167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a26e7-c614-41ac-b9c6-c58f29af6af9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:44.666167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a26e7-c614-41ac-b9c6-c58f29af6af9 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:44.668516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1161/2360] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:16:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:16:44.680996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be1df-849a-44af-be36-a279842508a3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:44.681942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be1df-849a-44af-be36-a279842508a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:44.681942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be1df-849a-44af-be36-a279842508a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:44.714177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-754be1df-849a-44af-be36-a279842508a3 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:44.715529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-754be1df-849a-44af-be36-a279842508a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:44.857213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1200/2361] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:44.867916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a26e7-c614-41ac-b9c6-c58f29af6af9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:44.870151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-697a26e7-c614-41ac-b9c6-c58f29af6af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:44.947756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a26e7-c614-41ac-b9c6-c58f29af6af9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:44.952296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b489a-85b9-412d-baa2-ac115be034f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:44.953219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b489a-85b9-412d-baa2-ac115be034f1 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:44.954089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b489a-85b9-412d-baa2-ac115be034f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:44.954566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b489a-85b9-412d-baa2-ac115be034f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:44.967273 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b489a-85b9-412d-baa2-ac115be034f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:44.969297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b489a-85b9-412d-baa2-ac115be034f1 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:44.991377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1162/2362] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:16:45.079945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd24cbc-3209-426c-9c29-ea0e8bdf2e96 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:45.080293 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd24cbc-3209-426c-9c29-ea0e8bdf2e96 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:45.081482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd24cbc-3209-426c-9c29-ea0e8bdf2e96 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:45.082908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1201/2363] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:16:45.088267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d598c70-4907-45ba-8b00-1547e3a13e4b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:45.088552 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d598c70-4907-45ba-8b00-1547e3a13e4b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:45.088827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d598c70-4907-45ba-8b00-1547e3a13e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:45.116351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4d598c70-4907-45ba-8b00-1547e3a13e4b None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:45.117598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4d598c70-4907-45ba-8b00-1547e3a13e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:45.196455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1163/2364] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:16:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:45.223498 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd24cbc-3209-426c-9c29-ea0e8bdf2e96 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:45.226160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8dd24cbc-3209-426c-9c29-ea0e8bdf2e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:45.238145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:45.246409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd24cbc-3209-426c-9c29-ea0e8bdf2e96 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:45.251992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e1bca-30bd-48b8-8765-be43eec24b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:45.253308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e1bca-30bd-48b8-8765-be43eec24b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:45.253308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e1bca-30bd-48b8-8765-be43eec24b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:45.253660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e1bca-30bd-48b8-8765-be43eec24b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:45.266099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e1bca-30bd-48b8-8765-be43eec24b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:45.267579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e1bca-30bd-48b8-8765-be43eec24b88 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:45.277290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1202/2365] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:45.365352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3138fe-9f7e-4ff0-b578-4c8f23c15e6c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:45.365679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3138fe-9f7e-4ff0-b578-4c8f23c15e6c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:45.365956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3138fe-9f7e-4ff0-b578-4c8f23c15e6c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:45.368101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1164/2366] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:16:45.373925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d887a3-5015-4f4f-b0b2-2e8fd055fff1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:45.374322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d887a3-5015-4f4f-b0b2-2e8fd055fff1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:45.374724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d887a3-5015-4f4f-b0b2-2e8fd055fff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:45.396035 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-76d887a3-5015-4f4f-b0b2-2e8fd055fff1 None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:45.397343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-76d887a3-5015-4f4f-b0b2-2e8fd055fff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:45.407265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1203/2367] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:16:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:16:45.418096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3138fe-9f7e-4ff0-b578-4c8f23c15e6c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:45.429504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3138fe-9f7e-4ff0-b578-4c8f23c15e6c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:45.433456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:45.434082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:45.434370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:45.434642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:45.446144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:45.447626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:45.459549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1165/2368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:46.221078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:46.221945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:46.221945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:46.242347 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:46.243740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:46.354426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1204/2369] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jul 28 01:16:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:46.362655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:46.365276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:46.496779 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddabf9-d166-4218-92fa-c85cb02134cc service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:46.506538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e7a8e-d6ce-4cc4-85fe-236a46ae56df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:46.507575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e7a8e-d6ce-4cc4-85fe-236a46ae56df service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:46.508023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e7a8e-d6ce-4cc4-85fe-236a46ae56df service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:46.508822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e7a8e-d6ce-4cc4-85fe-236a46ae56df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:46.529154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e7a8e-d6ce-4cc4-85fe-236a46ae56df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:46.533311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e7a8e-d6ce-4cc4-85fe-236a46ae56df service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:46.551795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1166/2370] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:16:46.590273 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:46.602815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854eba0-1eec-4619-80e2-5a305d217f75 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:46.614693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:46.615837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:46.616349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:46.616577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:46.621231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:46.781649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:46.783352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:46.801960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1205/2371] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:16:49.165518 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:49.166469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:49.166957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:49.212430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:49.220350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:49.231091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1167/2372] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:16:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:49.248720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:49.262587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f8ab-8790-4118-87c7-ebe4fd83eaff service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:49.269120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:49.269874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:49.270259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:49.270768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:49.283400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:49.285036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:49.295571 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1206/2373] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:16:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:16:49.754465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:49.756712 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:49.791413 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:49.894294 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3541021-5b13-45da-a8c1-783c3f434c74 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:49.899522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:49.900211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:49.900458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:49.900737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] PATH_INFO: `/v3/users/6f128878c21545e9aef8d9f9cb7b6881` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:49.904664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f128878c21545e9aef8d9f9cb7b6881)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:49.905436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:49.975720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f128878c21545e9aef8d9f9cb7b6881'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:49.993940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f128878c21545e9aef8d9f9cb7b6881'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:49.994558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f128878c21545e9aef8d9f9cb7b6881'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.032450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f128878c21545e9aef8d9f9cb7b6881'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.047314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f6c8276-7521-4f24-9dd7-0211920c565c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f128878c21545e9aef8d9f9cb7b6881'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.060423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1168/2374] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:16:49 2022] DELETE /identity/v3/users/6f128878c21545e9aef8d9f9cb7b6881 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:16:50.632651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:50.644831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4b62b-8675-4a89-b6ea-f857359d2965 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:50.747736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:50.748707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:50.749094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:50.749499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] PATH_INFO: `/v3/projects/0beae1c88e9948d28a386b5e31c68f85` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:50.750800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0beae1c88e9948d28a386b5e31c68f85)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:50.752853 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:50.786244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:50.855120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0beae1c88e9948d28a386b5e31c68f85', 'request_id': 'req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.862640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0beae1c88e9948d28a386b5e31c68f85', 'request_id': 'req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.880997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0beae1c88e9948d28a386b5e31c68f85', 'request_id': 'req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.903059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] The token cache is being invalidate because project 0beae1c88e9948d28a386b5e31c68f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:16:50.903528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.904206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1db2a13f-11cc-4c3c-83e3-2ac0a0b64a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:16:50.906927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1207/2375] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:16:50 2022] DELETE /identity/v3/projects/0beae1c88e9948d28a386b5e31c68f85 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:16:50.910718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:50.911579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:50.912001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:50.932826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:50.934060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:51.029919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1169/2376] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:51.031728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.032069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.032400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.053805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:51.055194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:51.063259 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1208/2377] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:16:51.067538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.078030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcf3b1-5d4c-4c90-8688-7d994d8a38b3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.084770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.085437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.085718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.085997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.086754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:16:51.087277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.088925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.107602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.112540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1170/2378] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:16:51.119831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a34e9-9cb1-4609-8fb4-b8079d4e755a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.125499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.126075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.126364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.126645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.127439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.129366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.174355 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1209/2379] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:16:51 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:16:51.180134 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.193103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85ad4-aa17-4792-8ac8-f945a963a735 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.220678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:16:51.270267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.270956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.271231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.271509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.272197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.274497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.312829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1171/2380] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:16:51 2022] POST /identity/v3/users => generated 285 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:16:51.319012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.335598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8247dd1-0e5c-48a6-b95b-667bced1f759 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.342141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.343019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.343412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.343881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.344923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.346672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.362973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1210/2381] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:16:51.367422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.380138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f15d-7847-4810-b367-73c8eaac9992 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.385640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560b8d9-ca1f-4e03-876e-22068c24a04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.386649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560b8d9-ca1f-4e03-876e-22068c24a04a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.391442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560b8d9-ca1f-4e03-876e-22068c24a04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.392031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560b8d9-ca1f-4e03-876e-22068c24a04a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.393041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560b8d9-ca1f-4e03-876e-22068c24a04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.395229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560b8d9-ca1f-4e03-876e-22068c24a04a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.413837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1172/2382] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:16:51.418203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.430176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f02af-3d84-451d-8ca4-82c0e834e1bc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.437129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.437885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.438177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.438462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] PATH_INFO: `/v3/projects/335f33129a2b4eba9c54b1b8083ca8c9/users/096b2b99a1444b71afe5689f681cabb5/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.459475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335f33129a2b4eba9c54b1b8083ca8c9, user_id=096b2b99a1444b71afe5689f681cabb5, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.462265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.485069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1211/2383] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:16:51 2022] PUT /identity/v3/projects/335f33129a2b4eba9c54b1b8083ca8c9/users/096b2b99a1444b71afe5689f681cabb5/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:16:51.492421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.493433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.494059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.555025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] MFA Rules not processed for user `096b2b99a1444b71afe5689f681cabb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:16:51.556685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:51.656236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1173/2384] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:16:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:16:51.668870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:16:51.671487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:16:51.781479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa438ac-4ecf-4db1-9d8f-d70f1a27bdc5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:16:51.791376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:16:51.792180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:16:51.792480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:16:51.792759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:16:51.803956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:16:51.805538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:16:51.818451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1212/2385] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:16:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:17:01.652230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:01.812177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:30.730244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:30.731910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:30.797690 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f32b3c-395f-4ea7-a30e-70105703c42e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:30.803506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:30.804447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:30.804741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:30.805030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:30.807836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:30.880805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:30.882251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a184-91b0-47eb-bed8-f8bc70a8f536 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:30.891853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1174/2386] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:17:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:17:33.783257 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:33.857592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefea3b-ddae-4b76-add8-93c6de57fa25 service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:33.862340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:33.863117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:33.863323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:33.863605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] PATH_INFO: `/v3/users/096b2b99a1444b71afe5689f681cabb5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:33.864798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=096b2b99a1444b71afe5689f681cabb5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:33.866738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:33.915797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096b2b99a1444b71afe5689f681cabb5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:33.916372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096b2b99a1444b71afe5689f681cabb5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:33.917037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096b2b99a1444b71afe5689f681cabb5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:33.924034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096b2b99a1444b71afe5689f681cabb5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:33.936771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096b2b99a1444b71afe5689f681cabb5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:33.951794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1213/2387] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:17:33 2022] DELETE /identity/v3/users/096b2b99a1444b71afe5689f681cabb5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:17:33.956439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:33.956959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:33.957131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:34.113937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:34.116370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:34.219625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1175/2388] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:17:34.230252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:34.231781 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:34.316522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945a3ab-bcec-4ad3-88a2-48ef74208960 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:34.340264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d47c6b-22a7-4362-8f9a-eb038df93246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:34.341078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47c6b-22a7-4362-8f9a-eb038df93246 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:34.341367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47c6b-22a7-4362-8f9a-eb038df93246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:34.341646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47c6b-22a7-4362-8f9a-eb038df93246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:34.352881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d47c6b-22a7-4362-8f9a-eb038df93246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:34.355271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d47c6b-22a7-4362-8f9a-eb038df93246 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:34.381956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1214/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:17:34.724968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:34.727211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:34.743854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c9a13-19b1-4c90-9178-991c527fd1aa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:34.750486 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:34.751323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:34.751698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:34.752072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] PATH_INFO: `/v3/projects/335f33129a2b4eba9c54b1b8083ca8c9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:34.753566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=335f33129a2b4eba9c54b1b8083ca8c9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:34.756130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:34.803608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:34.806694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335f33129a2b4eba9c54b1b8083ca8c9', 'request_id': 'req-44437cc3-9cc8-4485-a227-9a82e91f09b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:34.816578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335f33129a2b4eba9c54b1b8083ca8c9', 'request_id': 'req-44437cc3-9cc8-4485-a227-9a82e91f09b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:34.822210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335f33129a2b4eba9c54b1b8083ca8c9', 'request_id': 'req-44437cc3-9cc8-4485-a227-9a82e91f09b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:34.833495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] The token cache is being invalidate because project 335f33129a2b4eba9c54b1b8083ca8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:17:34.833915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:34.834651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44437cc3-9cc8-4485-a227-9a82e91f09b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:17:34.836485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1176/2390] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:17:34 2022] DELETE /identity/v3/projects/335f33129a2b4eba9c54b1b8083ca8c9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:17:34.841675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:34.842176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:34.842290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:34.862430 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:34.863698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:34.881704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:34.932311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1215/2391] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:17:34.939689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:34.939981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:34.940258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:34.961507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:34.962722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:34.970675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1177/2392] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:17:34.975643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:34.991703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a3d19-78e0-4f96-92de-947c3a1cd641 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.000204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.001111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.001486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.001885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.002990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:17:35.003457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.006334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.020627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1216/2393] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:17:35.025289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:35.035169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935ebb-a30b-4392-81d1-00d75a4b8977 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.040348 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.040967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.041250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.041526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.042266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.044494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.080301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1178/2394] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:17:35 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:17:35.085875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:35.098592 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adbf69-8dbe-492b-83c6-606f074fe0ce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.160299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.160951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.161235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.161506 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.162167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.164090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.196790 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1217/2395] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:17:35 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:17:35.201158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:35.212504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653d934-cfc7-4f9e-bf73-302b6c2f5350 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.218078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.218736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.219057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.219344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.220051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.221767 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.232693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1179/2396] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:17:35.238093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:35.249588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0a3b-ec33-4196-b12f-a95ee1145b4d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.255710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f11a1-05e5-4fa8-b5f7-73cf72d340fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.256341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f11a1-05e5-4fa8-b5f7-73cf72d340fd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.256620 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f11a1-05e5-4fa8-b5f7-73cf72d340fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.256891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f11a1-05e5-4fa8-b5f7-73cf72d340fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.257611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f11a1-05e5-4fa8-b5f7-73cf72d340fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.259251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f11a1-05e5-4fa8-b5f7-73cf72d340fd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.269902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1218/2397] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:17:35.274674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:35.285168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6bebd-730a-4e4c-9046-88adf7ff81ac admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.290122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.290794 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.291137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.291426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] PATH_INFO: `/v3/projects/b1331035c6654c0d9c8ba49f5a724f8c/users/eee7bd9e4fa14b159277977043f4813c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.306099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1331035c6654c0d9c8ba49f5a724f8c, user_id=eee7bd9e4fa14b159277977043f4813c, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.308887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.328215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1180/2398] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:17:35 2022] PUT /identity/v3/projects/b1331035c6654c0d9c8ba49f5a724f8c/users/eee7bd9e4fa14b159277977043f4813c/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:17:35.335126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cfa0e-740e-45e3-9af1-cb1fe370e75a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.335424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cfa0e-740e-45e3-9af1-cb1fe370e75a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.335699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cfa0e-740e-45e3-9af1-cb1fe370e75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.363174 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-359cfa0e-740e-45e3-9af1-cb1fe370e75a None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:35.364427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-359cfa0e-740e-45e3-9af1-cb1fe370e75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:35.457294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1219/2399] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:17:35.469430 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:35.472406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:35.533666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf9575-eb0f-45b6-8ffa-0841663994e2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:35.538732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:35.539339 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:35.539637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:35.539913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:35.549990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:35.551680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:35.567049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1181/2400] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:17:40.246125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:40.246626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:40.246743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:40.308793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] MFA Rules not processed for user `eee7bd9e4fa14b159277977043f4813c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:40.317570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:40.374158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:40.484789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:40.489921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1220/2401] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:17:40.509969 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153352-19ca-4aea-ae1e-a0a17195db5d service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:40.522691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:40.523854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:40.524302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:40.524752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:40.525627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:40.527992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:40.580564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1182/2402] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:17:40 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:17:40.584651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:40.598444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95d35b-c56d-4091-99b1-87e7655cad75 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:40.774783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:40.775962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:40.776364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:40.776558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:40.777227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:40.779328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:40.825075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1221/2403] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:17:40 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:17:40.829189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:40.843505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce56c5c-a7be-4676-aa61-07ae1a5a07ee admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:40.850023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:40.850822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:40.851135 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:40.851420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:40.852137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:40.853791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:40.866187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1183/2404] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:17:40.870126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:40.881234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf21c5-f6b1-4ae7-be6f-4816c4298afd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:40.889575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:40.890324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:40.890669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:40.890999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] PATH_INFO: `/v3/projects/e1b1f212ac024c8cab7ecf606e39179f/users/94ccc165e7754ecbb29bf6b9c854a568/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:40.909204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b1f212ac024c8cab7ecf606e39179f, user_id=94ccc165e7754ecbb29bf6b9c854a568, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:40.911913 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:40.926550 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1222/2405] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:17:40 2022] PUT /identity/v3/projects/e1b1f212ac024c8cab7ecf606e39179f/users/94ccc165e7754ecbb29bf6b9c854a568/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:17:40.930937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:40.941986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075631bc-4ff8-46e1-a718-270c5e6162cb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:41.005241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:41.034828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b038-2e64-4a6c-8dcd-b5971709a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:41.035502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b038-2e64-4a6c-8dcd-b5971709a33f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:41.035778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b038-2e64-4a6c-8dcd-b5971709a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:41.036045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b038-2e64-4a6c-8dcd-b5971709a33f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:41.036786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b038-2e64-4a6c-8dcd-b5971709a33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:41.038406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b038-2e64-4a6c-8dcd-b5971709a33f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:41.049594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1184/2406] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:17:41.054118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:41.065991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44328ba-56ff-4420-ba3c-ec39be6360c0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:41.071217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:41.071935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:41.072222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:41.072501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] PATH_INFO: `/v3/projects/e1b1f212ac024c8cab7ecf606e39179f/users/94ccc165e7754ecbb29bf6b9c854a568/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:41.074825 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b1f212ac024c8cab7ecf606e39179f, user_id=94ccc165e7754ecbb29bf6b9c854a568, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:41.077535 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:41.094373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1223/2407] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:17:41 2022] PUT /identity/v3/projects/e1b1f212ac024c8cab7ecf606e39179f/users/94ccc165e7754ecbb29bf6b9c854a568/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:17:42.197695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58012f4-bcdf-4cd2-bf63-e13e194ae70d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:42.198548 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58012f4-bcdf-4cd2-bf63-e13e194ae70d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:42.199051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58012f4-bcdf-4cd2-bf63-e13e194ae70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:42.244936 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c58012f4-bcdf-4cd2-bf63-e13e194ae70d None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:42.246278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c58012f4-bcdf-4cd2-bf63-e13e194ae70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:42.363050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1185/2408] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jul 28 01:17:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:17:42.393726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:42.395793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:42.545108 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d4752-b5c3-459b-a665-a8e27ab2de00 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:42.549139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:42.552771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:42.552771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:42.552771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:42.560722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:42.562877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:42.569019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:42.581172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1224/2409] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:17:46.543463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929694f-13a0-4924-82d1-6c4853a042e4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:46.544051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929694f-13a0-4924-82d1-6c4853a042e4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:46.544051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929694f-13a0-4924-82d1-6c4853a042e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:46.577557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6929694f-13a0-4924-82d1-6c4853a042e4 None None] MFA Rules not processed for user `94ccc165e7754ecbb29bf6b9c854a568`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:46.578883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6929694f-13a0-4924-82d1-6c4853a042e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:46.655833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1186/2410] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:17:46.669137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:17:46.670681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:46.772522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de366e-3e8c-4399-b5d8-3083e4ed3683 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:17:46.781288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3dce4-c598-401b-97d6-77a5d6a54a64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:17:46.782071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3dce4-c598-401b-97d6-77a5d6a54a64 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:46.782356 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3dce4-c598-401b-97d6-77a5d6a54a64 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:46.782804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3dce4-c598-401b-97d6-77a5d6a54a64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:46.843669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3dce4-c598-401b-97d6-77a5d6a54a64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:17:46.845362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3dce4-c598-401b-97d6-77a5d6a54a64 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:17:46.856278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1225/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:17:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:17:48.288741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:17:48.289980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:17:48.290417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:17:48.319613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] MFA Rules not processed for user `eee7bd9e4fa14b159277977043f4813c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:17:48.321043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:17:48.331882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1187/2412] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:17:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:17:56.852847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=56 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:17:58.309186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=73 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:18:08.946721 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:08.947343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:08.947598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:08.968422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] MFA Rules not processed for user `018eb80709e94d5e8653fb6eeb342c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:18:08.969775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:09.036211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1226/2413] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:18:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:18:19.027677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=25 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:18:28.652055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:28.653646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:28.716613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fe55f-d76d-4782-9cb2-66338d6dc90e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:28.720624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:28.721368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:28.721719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:28.722057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:28.732840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:28.735059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:28.748858 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1188/2414] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:29.714625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:29.724951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf5fe3-dc0d-4776-92b9-0652b17d94ec None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:29.729527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:29.730187 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:29.730473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:29.730749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:29.732859 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:29.748523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:29.749972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7c235-a40d-4ff3-875c-96c117d951c4 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:29.763812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1227/2415] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:32.204204 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:32.213167 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fb076-6736-4983-b116-af315fce1891 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:32.216792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:32.217651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:32.217922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:32.218206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:32.220006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:32.234160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:32.235653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:32.247600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1189/2416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:35.587887 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:35.588442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:35.588442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:35.609345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] MFA Rules not processed for user `166cef282cb24a5aaef0ca8b28f02e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:18:35.610562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:35.677039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1228/2417] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:18:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:18:36.364537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:36.366236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:36.442576 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042dfb47-f2ee-424b-ae1a-4a66682cb429 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:36.447803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:36.448433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:36.448717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:36.448995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:36.451900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:36.463586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:36.465235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549ea7f-b018-432c-813e-511ce273b356 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:36.475825 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1190/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:45.668996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=31 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:18:46.471199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=64 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:18:50.916411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:50.928597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e31b69-168b-4c55-ae6e-709a96e83633 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:50.932945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:50.933605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:50.933963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:50.934316 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:50.936303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:50.951395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:50.953266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:50.963480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1229/2419] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:51.234326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:51.234898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:51.235012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:51.259525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] MFA Rules not processed for user `c8ab1df6f15143de8f922a626317fa9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:18:51.261060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:51.326288 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1191/2420] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:18:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:18:51.714222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:51.723043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0791543-e087-4543-9738-590f1a97b585 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:51.727146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:51.727782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:51.728069 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:51.728350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:51.730169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:51.741623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:51.743315 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:51.752525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1230/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:53.110865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:53.121507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c2177-0c01-4923-a707-57270c5ed9c5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:53.125504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:53.126145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:53.126438 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:53.126721 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:53.128813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:53.141666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:53.143450 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:53.147167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:53.149865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:53.158205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1192/2422] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:53.167815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60c62a-c729-4a56-89f0-393784c452c7 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:53.182562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:53.183651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:53.184229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:53.184517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:53.194980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:53.196631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:53.207255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1231/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:54.714748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:54.716285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:54.807815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98fffb-c8f2-4de4-82fa-62ca4c7fa300 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:54.813733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:54.814391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:54.814678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:54.815337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:54.817173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:54.829300 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:54.833076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11843959-a3da-48ff-abb6-64269eb4dc91 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:54.844322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1193/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:18:56.817589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:18:56.829811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aa5f7-fbb5-4d8e-8a9e-09285f436e52 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:18:56.836772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:18:56.837410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:18:56.837695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:18:56.837976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:18:56.840791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:18:56.855127 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:18:56.856337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:18:56.869662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1232/2425] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:18:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:04.838807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=57 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:06.863530 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=24 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:07.797088 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5be6d2-dce8-4acc-a30c-1cd8c7ae555b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:07.797554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5be6d2-dce8-4acc-a30c-1cd8c7ae555b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:07.798693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5be6d2-dce8-4acc-a30c-1cd8c7ae555b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:07.819889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8c5be6d2-dce8-4acc-a30c-1cd8c7ae555b None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:07.821710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c5be6d2-dce8-4acc-a30c-1cd8c7ae555b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:07.829330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1194/2426] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jul 28 01:19:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:07.880428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:07.893015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926abe6-7850-47a4-9def-45bc02ac7fdb service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:07.899382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8cc9d-c23a-43ec-9acf-320003d8704c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:07.900046 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8cc9d-c23a-43ec-9acf-320003d8704c service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:07.900389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8cc9d-c23a-43ec-9acf-320003d8704c service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:07.900687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8cc9d-c23a-43ec-9acf-320003d8704c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:07.909921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8cc9d-c23a-43ec-9acf-320003d8704c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:07.911587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8cc9d-c23a-43ec-9acf-320003d8704c service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:07.920270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1233/2427] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:19:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:19:12.524882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffad90-6a74-423f-9ebb-9557e6840562 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:12.526149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffad90-6a74-423f-9ebb-9557e6840562 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:12.526433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffad90-6a74-423f-9ebb-9557e6840562 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:12.527874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1195/2428] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:19:12.565490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:12.566016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:12.566016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:12.586513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] MFA Rules not processed for user `45f29692bb444492a58051889f4f44b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:12.587785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:12.597934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1234/2429] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:19:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:12.614209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffad90-6a74-423f-9ebb-9557e6840562 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:12.631032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffad90-6a74-423f-9ebb-9557e6840562 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:12.636879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:12.637614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:12.637965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:12.638315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:12.647521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:12.649208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:12.659398 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1196/2430] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:16.204631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:16.206146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:16.275243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7697ad-ca61-4cac-a958-989967b6db4e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:16.280551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:16.281326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:16.281905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:16.282212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] PATH_INFO: `/v3/users/70b75ae15822428f916c3403f9a67225` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:16.283773 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b75ae15822428f916c3403f9a67225)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:16.285741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:16.338910 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b75ae15822428f916c3403f9a67225'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.339594 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b75ae15822428f916c3403f9a67225'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.340274 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b75ae15822428f916c3403f9a67225'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.348121 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b75ae15822428f916c3403f9a67225'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.364015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b75ae15822428f916c3403f9a67225'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.384695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1235/2431] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:16 2022] DELETE /identity/v3/users/70b75ae15822428f916c3403f9a67225 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:16.593686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:16.606082 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e35c6-8b40-42ea-8059-6bb43f324c8e service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:16.668654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:16.669312 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:16.669589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:16.669865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] PATH_INFO: `/v3/projects/9889594f4bf84c4591714b505f1861fe` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:16.671091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9889594f4bf84c4591714b505f1861fe)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:16.673057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:16.732407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9889594f4bf84c4591714b505f1861fe', 'request_id': 'req-f7f4e751-f7a2-45bf-862f-208f32af80d4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.741434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9889594f4bf84c4591714b505f1861fe', 'request_id': 'req-f7f4e751-f7a2-45bf-862f-208f32af80d4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.747325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9889594f4bf84c4591714b505f1861fe', 'request_id': 'req-f7f4e751-f7a2-45bf-862f-208f32af80d4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.759094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] The token cache is being invalidate because project 9889594f4bf84c4591714b505f1861fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:16.759392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.760128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.761799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1197/2432] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:16 2022] DELETE /identity/v3/projects/9889594f4bf84c4591714b505f1861fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:16.766113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:16.767801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:16.781840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05175c8-b5f6-4088-8f9c-7f1619b56ee2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:16.837709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:16.838338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:16.838874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:16.839181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] PATH_INFO: `/v3/users/35cf24f2df094b09be654f8d18b8c99a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:16.840414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35cf24f2df094b09be654f8d18b8c99a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:16.842354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:16.843946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:16.898929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35cf24f2df094b09be654f8d18b8c99a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.899280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35cf24f2df094b09be654f8d18b8c99a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.899939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35cf24f2df094b09be654f8d18b8c99a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.908129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35cf24f2df094b09be654f8d18b8c99a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.920363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35cf24f2df094b09be654f8d18b8c99a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:16.933669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1236/2433] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:16 2022] DELETE /identity/v3/users/35cf24f2df094b09be654f8d18b8c99a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:17.126597 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:17.128720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:17.150503 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4e751-f7a2-45bf-862f-208f32af80d4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:17.171147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:17.173013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:17.228069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:17.228729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:17.229066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:17.229343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] PATH_INFO: `/v3/projects/f0b12ede1da04e0d88ecb4dd72e74afa` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:17.230859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b12ede1da04e0d88ecb4dd72e74afa)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:17.232780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:17.233607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c5cf-368b-4528-8eb1-052c8ff38c0f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:17.237221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:17.237856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:17.238143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:17.238426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:17.240241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:17.244534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:17.253031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:17.254520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:17.265838 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1237/2434] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:17.283287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b12ede1da04e0d88ecb4dd72e74afa', 'request_id': 'req-14c7523e-9c85-4de8-905e-5ac2abe849b3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:17.292049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b12ede1da04e0d88ecb4dd72e74afa', 'request_id': 'req-14c7523e-9c85-4de8-905e-5ac2abe849b3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:17.298505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b12ede1da04e0d88ecb4dd72e74afa', 'request_id': 'req-14c7523e-9c85-4de8-905e-5ac2abe849b3'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:17.310040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] The token cache is being invalidate because project f0b12ede1da04e0d88ecb4dd72e74afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:17.310330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:17.311085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:17.312777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1198/2435] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:17 2022] DELETE /identity/v3/projects/f0b12ede1da04e0d88ecb4dd72e74afa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:20.628820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:20.630669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:20.643963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451ae36-f808-447f-9f9f-d2ea0c6c8531 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:20.705051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:20.705705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:20.705991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:20.706269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] PATH_INFO: `/v3/users/018eb80709e94d5e8653fb6eeb342c54` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:20.707517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018eb80709e94d5e8653fb6eeb342c54)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:20.709463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:20.731978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:20.765606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018eb80709e94d5e8653fb6eeb342c54'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:20.766267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018eb80709e94d5e8653fb6eeb342c54'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:20.767007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018eb80709e94d5e8653fb6eeb342c54'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:20.775040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018eb80709e94d5e8653fb6eeb342c54'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:20.790705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-15e22975-8e8f-4429-9c67-d830bea2d811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018eb80709e94d5e8653fb6eeb342c54'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:20.810092 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1238/2436] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:20 2022] DELETE /identity/v3/users/018eb80709e94d5e8653fb6eeb342c54 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:20.990927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:20.992412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:21.006176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7523e-9c85-4de8-905e-5ac2abe849b3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.067943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.068591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.068873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.069156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] PATH_INFO: `/v3/projects/af4689d2f0ef4d71a1c465610011f2d7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.070303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4689d2f0ef4d71a1c465610011f2d7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.072244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.125102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4689d2f0ef4d71a1c465610011f2d7', 'request_id': 'req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:21.135192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4689d2f0ef4d71a1c465610011f2d7', 'request_id': 'req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:21.141330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4689d2f0ef4d71a1c465610011f2d7', 'request_id': 'req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:21.156065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] The token cache is being invalidate because project af4689d2f0ef4d71a1c465610011f2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:21.156376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:21.157060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ca0af32-edd9-4ed8-ab53-75ea872d0cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:21.158626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1199/2437] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:20 2022] DELETE /identity/v3/projects/af4689d2f0ef4d71a1c465610011f2d7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:21.163906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.164221 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.164477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.184707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:21.186617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:21.250968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1239/2438] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:21.259769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.260053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.260325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.281281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:21.282158 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:21.292125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1200/2439] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:21.296681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.306920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cf13-98d2-4abb-b6b2-6d5d228384b2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.313116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.313797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.314025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.314296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.315106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:19:21.315412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.317014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.329689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1240/2440] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:19:21.334301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.343854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1b51-c3ee-41ce-b5b3-5a8d15d1a4bc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.350093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.350724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.351051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.351330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.352045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.353999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.385613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1201/2441] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:21.390309 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.400538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea997c-1f9c-466a-b23e-d15079c8863a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.461868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.462513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.462793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.463100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.463771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.465704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.488851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1241/2442] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:21.493120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.498537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:21.505695 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d17f9-e404-4b64-b41a-9420cf44cadc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.512251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.512916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.513578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.513874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.514613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.516285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.527911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1202/2443] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:21.532282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.542813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142967f3-e235-4d4e-9df7-4bdb6931eb6d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.548082 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15262e85-370b-441a-a64f-6bada22f936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.548717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15262e85-370b-441a-a64f-6bada22f936d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.548995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15262e85-370b-441a-a64f-6bada22f936d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.549275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15262e85-370b-441a-a64f-6bada22f936d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.549982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15262e85-370b-441a-a64f-6bada22f936d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.551625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15262e85-370b-441a-a64f-6bada22f936d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.563123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1242/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:21.569412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.580993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b3e85-2b5a-4549-b989-5b0a3694dc67 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.589265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.590318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.590599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.590896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] PATH_INFO: `/v3/projects/744a5e43ac5d4c6bb16ab5d5bea8c930/users/4e862aa6f5504aff974d2e44ceb05ed3/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.606789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] RBAC: Authorizing `identity:create_grant(project_id=744a5e43ac5d4c6bb16ab5d5bea8c930, user_id=4e862aa6f5504aff974d2e44ceb05ed3, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.609394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.624903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1203/2445] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:21 2022] PUT /identity/v3/projects/744a5e43ac5d4c6bb16ab5d5bea8c930/users/4e862aa6f5504aff974d2e44ceb05ed3/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:21.635830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.636371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.636371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.650225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:21.667073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] MFA Rules not processed for user `4e862aa6f5504aff974d2e44ceb05ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:21.668289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:21.739416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1243/2446] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:21.744207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.812777 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c983f-dc5d-4f8d-8e2e-9b519040600a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.817494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.818113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.818397 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.818669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.819369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.821235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.854471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1204/2447] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:21.860603 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.876131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9375481-91dd-43db-b880-0a1f8d7e4af8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.938999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.939640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.939917 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.940208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.940879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.942828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:21.966000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1244/2448] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:21 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:21.970325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:21.981488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e560a-8562-4feb-a456-5e52ba1642fd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:21.986507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:21.987240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:21.987528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:21.987801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:21.988512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:21.990101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:22.001183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1205/2449] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:22.006206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:22.017749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0ea0f-def3-4671-aae0-8adee6870f7d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:22.023890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:22.024584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:22.024877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:22.025158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] PATH_INFO: `/v3/projects/c38e74588ff946308b0b1bbe9f23045d/users/ede5ef8c6a054dffa2bdde40c56cfb32/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:22.052488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38e74588ff946308b0b1bbe9f23045d, user_id=ede5ef8c6a054dffa2bdde40c56cfb32, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:22.055342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:22.071796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1245/2450] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:22 2022] PUT /identity/v3/projects/c38e74588ff946308b0b1bbe9f23045d/users/ede5ef8c6a054dffa2bdde40c56cfb32/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:22.076195 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:22.087264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc176-3234-42db-82b6-9c3b1b1485b4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:22.152512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de5914-b976-421b-8c35-e82cacd761a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:22.153182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de5914-b976-421b-8c35-e82cacd761a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:22.153466 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de5914-b976-421b-8c35-e82cacd761a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:22.153742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de5914-b976-421b-8c35-e82cacd761a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:22.154446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de5914-b976-421b-8c35-e82cacd761a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:22.156087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de5914-b976-421b-8c35-e82cacd761a9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:22.165956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1206/2451] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:22.170473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:22.184105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce73a2e-67b9-4c7a-a059-c39935e71b63 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:22.185561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:22.186241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:22.186529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:22.186803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] PATH_INFO: `/v3/projects/c38e74588ff946308b0b1bbe9f23045d/users/ede5ef8c6a054dffa2bdde40c56cfb32/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:22.188798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38e74588ff946308b0b1bbe9f23045d, user_id=ede5ef8c6a054dffa2bdde40c56cfb32, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:22.191509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:22.203434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1246/2452] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:22 2022] PUT /identity/v3/projects/c38e74588ff946308b0b1bbe9f23045d/users/ede5ef8c6a054dffa2bdde40c56cfb32/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:22.215669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:22.215965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:22.216264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:22.245023 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] MFA Rules not processed for user `ede5ef8c6a054dffa2bdde40c56cfb32`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:22.246264 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:22.272411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:22.321696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1207/2453] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:22 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:22.333511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:22.335507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:22.400863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:22.405955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b454-4d50-4cd7-9aa3-db60e493c024 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:22.412301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:22.412945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:22.413234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:22.413517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:22.423267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:22.424738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:22.434518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1247/2454] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:22 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:22.482508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:22.484164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:22.498033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c2939-f049-4567-ac63-1b3ca2f5d570 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:22.503030 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e455c-9223-471f-9ac5-f964546cd813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:22.503661 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e455c-9223-471f-9ac5-f964546cd813 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:22.503943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e455c-9223-471f-9ac5-f964546cd813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:22.504227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e455c-9223-471f-9ac5-f964546cd813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:22.565988 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e455c-9223-471f-9ac5-f964546cd813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:22.567478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e455c-9223-471f-9ac5-f964546cd813 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:22.577467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1208/2455] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:26.084059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:26.154562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67f625-4312-4992-888b-e8e034b61023 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:26.160593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:26.161280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:26.161703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:26.161987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] PATH_INFO: `/v3/users/4e862aa6f5504aff974d2e44ceb05ed3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:26.163286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e862aa6f5504aff974d2e44ceb05ed3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:26.165236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:26.222159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e862aa6f5504aff974d2e44ceb05ed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.222903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e862aa6f5504aff974d2e44ceb05ed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.223531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e862aa6f5504aff974d2e44ceb05ed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.230779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e862aa6f5504aff974d2e44ceb05ed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.243252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e862aa6f5504aff974d2e44ceb05ed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.257500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1248/2456] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:26 2022] DELETE /identity/v3/users/4e862aa6f5504aff974d2e44ceb05ed3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:26.262159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:26.262650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:26.262800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:26.283621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:26.284956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:26.349064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1209/2457] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:26.359342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:26.361061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:26.422636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02ab53-0833-436a-b969-a93ab7b239dd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:26.426708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:26.427450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:26.427738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:26.428017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:26.437959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:26.439477 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:26.455018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1249/2458] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:26.802224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:26.804363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:26.820284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16357059-4af2-490c-af28-f35c96ff3319 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:26.832441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:26.832892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:26.833188 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:26.833472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] PATH_INFO: `/v3/projects/744a5e43ac5d4c6bb16ab5d5bea8c930` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:26.834790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744a5e43ac5d4c6bb16ab5d5bea8c930)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:26.837064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:26.886912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744a5e43ac5d4c6bb16ab5d5bea8c930', 'request_id': 'req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.900774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744a5e43ac5d4c6bb16ab5d5bea8c930', 'request_id': 'req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.902674 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:26.908826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744a5e43ac5d4c6bb16ab5d5bea8c930', 'request_id': 'req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.925488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] The token cache is being invalidate because project 744a5e43ac5d4c6bb16ab5d5bea8c930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:26.925488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.925488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-cbd3a7aa-30ea-49bf-bcb9-59c423b79e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:26.930164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1210/2459] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:26 2022] DELETE /identity/v3/projects/744a5e43ac5d4c6bb16ab5d5bea8c930 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:26.940943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:26.949026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:26.949377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:26.949662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:26.953750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa6be0-b151-4c66-98a7-7fb3ce3c2b45 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:26.964002 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:26.975994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] MFA Rules not processed for user `d9cb42db2f9a4b80b17bef0a86945a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:26.977231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:27.028967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.029636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.029924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.030218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] PATH_INFO: `/v3/users/ede5ef8c6a054dffa2bdde40c56cfb32` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.031507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede5ef8c6a054dffa2bdde40c56cfb32)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.033504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.042112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1211/2460] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:27.084871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede5ef8c6a054dffa2bdde40c56cfb32'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.085544 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede5ef8c6a054dffa2bdde40c56cfb32'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.086191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede5ef8c6a054dffa2bdde40c56cfb32'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.093560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede5ef8c6a054dffa2bdde40c56cfb32'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.105828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede5ef8c6a054dffa2bdde40c56cfb32'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.120487 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1250/2461] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:26 2022] DELETE /identity/v3/users/ede5ef8c6a054dffa2bdde40c56cfb32 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:27.376700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.378555 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:27.404619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.406099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:27.450026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94583-44c7-485b-bf52-efa5cb23bd1b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.483122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c793aeb-1c9f-4323-b808-35dd30aeff24 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.487813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.488479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.488793 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.489205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] PATH_INFO: `/v3/projects/c38e74588ff946308b0b1bbe9f23045d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.490412 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38e74588ff946308b0b1bbe9f23045d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.492373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.504325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.505643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.506142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.506570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.508641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:27.519922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.521394 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c41140-d290-44e0-8643-6a13fdd18feb service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.531589 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1212/2462] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:27.541702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:27.554931 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38e74588ff946308b0b1bbe9f23045d', 'request_id': 'req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.561982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38e74588ff946308b0b1bbe9f23045d', 'request_id': 'req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.574798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38e74588ff946308b0b1bbe9f23045d', 'request_id': 'req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.592340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] The token cache is being invalidate because project c38e74588ff946308b0b1bbe9f23045d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:27.592694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.593433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c664f0f5-2f1b-4443-8c1b-4dfcfa8c5137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:27.595138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1251/2463] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:27 2022] DELETE /identity/v3/projects/c38e74588ff946308b0b1bbe9f23045d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:19:27.601182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.601475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.601752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.624194 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:27.625447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:27.673138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:27.690291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1213/2464] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:27.698482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.698875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.699234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.720735 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:27.721818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:27.730936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1252/2465] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:27.735571 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.746104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a410e0c-113a-4266-aab9-223592488e73 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.750960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.751592 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.751893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.752171 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.752882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:19:27.753184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.755139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.767390 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1214/2466] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:19:27.771964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.781416 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b44ef-ac68-4987-af3f-147acf3f3322 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.786228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.786883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.787167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.787444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.788122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.790057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.830559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1253/2467] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:27 2022] POST /identity/v3/projects => generated 407 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:27.835359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.846912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8baec2-ed75-4f57-8124-449fcad5bb31 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.911483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.912133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.912419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.912697 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.913374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.915366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.937855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1215/2468] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:27 2022] POST /identity/v3/users => generated 304 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:19:27.942485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.953501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4e47-42a0-41cd-ad33-1b721f3f378f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.959452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:27.960118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:27.960402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:27.960677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:27.961417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:27.963062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:27.977241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1254/2469] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:27.978308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:27.993562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3cba5-b997-4b3c-90cc-fdd4fe34ab3a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:27.999899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e7ecd-66ee-49ee-8698-57cde1da4b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.000691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e7ecd-66ee-49ee-8698-57cde1da4b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.001073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e7ecd-66ee-49ee-8698-57cde1da4b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.001494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e7ecd-66ee-49ee-8698-57cde1da4b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.002554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e7ecd-66ee-49ee-8698-57cde1da4b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.005828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e7ecd-66ee-49ee-8698-57cde1da4b05 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.018630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1216/2470] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:19:28.023018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.034985 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b42bb-a69b-4c45-a93a-f7ebee8ba3a3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.039807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.040493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.040777 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.041072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] PATH_INFO: `/v3/projects/7a0466cde0934175bce70118d73e9a5b/users/8b2f9c72aab04c98bec61317f902ff8e/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.057617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0466cde0934175bce70118d73e9a5b, user_id=8b2f9c72aab04c98bec61317f902ff8e, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.060348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.075852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1255/2471] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:28 2022] PUT /identity/v3/projects/7a0466cde0934175bce70118d73e9a5b/users/8b2f9c72aab04c98bec61317f902ff8e/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:19:28.087400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.087675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.087955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.118187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] MFA Rules not processed for user `8b2f9c72aab04c98bec61317f902ff8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:28.119811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:28.192532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1217/2472] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:28.197190 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.267054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54d7bc-0307-4a96-a7da-d5d72be55ce7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.272458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.273123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.273409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.273688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.274380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.276631 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.310026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1256/2473] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:28 2022] POST /identity/v3/projects => generated 409 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:28.314271 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.325437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5968c0-2dbb-4a03-b403-7e069b743869 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.369732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:28.386141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.386849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.387121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.387404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.388152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.390101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.412255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1218/2474] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:28 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:28.416516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.428063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe08a9-232a-4ba6-8872-418d000b34ec admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.433116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.433811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.434103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.434446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.435160 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.436776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.449455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1257/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:28.453881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.465149 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394d17-a23d-48ae-b89b-a36acef3b280 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.476351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.477064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.477372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.477651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] PATH_INFO: `/v3/projects/55e29404611343ec963776111c2594af/users/9627d3fa312843d5a1a74182d91ebe1e/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.494563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e29404611343ec963776111c2594af, user_id=9627d3fa312843d5a1a74182d91ebe1e, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.497327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.513686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1219/2476] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:28 2022] PUT /identity/v3/projects/55e29404611343ec963776111c2594af/users/9627d3fa312843d5a1a74182d91ebe1e/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:28.518137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.530119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b841da-56c4-4eb8-8797-d020d6be0c4a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.566042 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:28.598463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f04a89-2df9-4d69-b8a2-f966d495ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.599222 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f04a89-2df9-4d69-b8a2-f966d495ceb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.599509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f04a89-2df9-4d69-b8a2-f966d495ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.599791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f04a89-2df9-4d69-b8a2-f966d495ceb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.600499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f04a89-2df9-4d69-b8a2-f966d495ceb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.602105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f04a89-2df9-4d69-b8a2-f966d495ceb4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.612900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1258/2477] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:28.617263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.628846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa65a03-595a-4ce3-936b-fa2d6b20925b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.634567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.635311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.635667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.639214 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] PATH_INFO: `/v3/projects/55e29404611343ec963776111c2594af/users/9627d3fa312843d5a1a74182d91ebe1e/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.642693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e29404611343ec963776111c2594af, user_id=9627d3fa312843d5a1a74182d91ebe1e, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.645441 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.656166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1220/2478] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:28 2022] PUT /identity/v3/projects/55e29404611343ec963776111c2594af/users/9627d3fa312843d5a1a74182d91ebe1e/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:28.667517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.667936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.668297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.697059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] MFA Rules not processed for user `9627d3fa312843d5a1a74182d91ebe1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:28.698647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:28.773697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1259/2479] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:28.788401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:28.790248 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:28.851783 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967784-4b12-48d8-964c-c81e0b0127ab admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:28.855460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:28.856090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:28.856379 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:28.856654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:28.866778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:28.868338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:28.878786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1221/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:29.076369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:29.091284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aba62-e5e5-4998-a33d-71032849f7d8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:29.097559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:29.098573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:29.098914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:29.099180 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:29.186295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:29.188000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:29.201936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1260/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:29 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:34.702332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:34.704064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:34.780284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b963-4da5-442f-97e0-82933727ef88 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:34.786196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:34.786829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:34.787152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:34.787442 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:34.789314 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:34.825049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:34.863899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:34.865506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635cc20-6fdc-4cc6-b6a7-3ef32033a022 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:34.875087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1222/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:37.739311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:37.798014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:37.815439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed82cbe-4a0a-46ec-b0f1-e76f48aa65df service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:37.823417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:37.824400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:37.824800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:37.825094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] PATH_INFO: `/v3/users/8b2f9c72aab04c98bec61317f902ff8e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:37.826725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2f9c72aab04c98bec61317f902ff8e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:37.829691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:37.877998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2f9c72aab04c98bec61317f902ff8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:37.878655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2f9c72aab04c98bec61317f902ff8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:37.879295 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2f9c72aab04c98bec61317f902ff8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:37.886676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2f9c72aab04c98bec61317f902ff8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:37.899088 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2f9c72aab04c98bec61317f902ff8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:37.913552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1261/2483] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:37 2022] DELETE /identity/v3/users/8b2f9c72aab04c98bec61317f902ff8e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:37.918939 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:37.919504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:37.919504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:37.939221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:37.940535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:38.009754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1223/2484] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:38.020599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:38.022130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:38.082876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a06a9-7a98-41fe-aabe-480fd7108b2f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:38.086807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:38.087472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:38.087760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:38.088040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:38.097760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:38.099256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:38.108916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1262/2485] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:38.425846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:38.427342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:38.439665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e0e5b-3ff0-4e30-8af4-7d2f78148d42 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:38.449407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:38.449407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:38.449407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:38.449407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] PATH_INFO: `/v3/projects/7a0466cde0934175bce70118d73e9a5b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:38.450132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0466cde0934175bce70118d73e9a5b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:38.450132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:38.499916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0466cde0934175bce70118d73e9a5b', 'request_id': 'req-619bc0f9-754a-4860-a8df-5265d31ea4db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.509993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0466cde0934175bce70118d73e9a5b', 'request_id': 'req-619bc0f9-754a-4860-a8df-5265d31ea4db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.518357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0466cde0934175bce70118d73e9a5b', 'request_id': 'req-619bc0f9-754a-4860-a8df-5265d31ea4db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.530121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] The token cache is being invalidate because project 7a0466cde0934175bce70118d73e9a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:38.530333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.531297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.532947 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1224/2486] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:38 2022] DELETE /identity/v3/projects/7a0466cde0934175bce70118d73e9a5b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:38.538266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:38.548920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a561c-6955-41fb-b174-1d07ecb7adce service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:38.608496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:38.609143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:38.609428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:38.609707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] PATH_INFO: `/v3/users/9627d3fa312843d5a1a74182d91ebe1e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:38.610954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9627d3fa312843d5a1a74182d91ebe1e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:38.613131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:38.631467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:38.669028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9627d3fa312843d5a1a74182d91ebe1e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.669643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9627d3fa312843d5a1a74182d91ebe1e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.670317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9627d3fa312843d5a1a74182d91ebe1e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.678393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9627d3fa312843d5a1a74182d91ebe1e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.691311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7fdf1e47-20b8-4eb8-9539-05b373cfc80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9627d3fa312843d5a1a74182d91ebe1e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:38.705532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1263/2487] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:38 2022] DELETE /identity/v3/users/9627d3fa312843d5a1a74182d91ebe1e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:19:38.994072 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:38.995628 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:39.013479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bc0f9-754a-4860-a8df-5265d31ea4db admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.073009 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.073584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.073872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.074200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] PATH_INFO: `/v3/projects/55e29404611343ec963776111c2594af` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.075404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e29404611343ec963776111c2594af)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.077420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.078986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:39.136461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e29404611343ec963776111c2594af', 'request_id': 'req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:39.146620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e29404611343ec963776111c2594af', 'request_id': 'req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:39.152444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e29404611343ec963776111c2594af', 'request_id': 'req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:39.165775 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] The token cache is being invalidate because project 55e29404611343ec963776111c2594af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:39.166078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:39.167177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-01c66e10-e7eb-4d88-8556-054d8a9b8ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:39.169109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1225/2488] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:38 2022] DELETE /identity/v3/projects/55e29404611343ec963776111c2594af => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:19:39.177109 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783672f4-d029-40f6-ae34-6033f265106d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.177610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783672f4-d029-40f6-ae34-6033f265106d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.178018 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783672f4-d029-40f6-ae34-6033f265106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.208528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-783672f4-d029-40f6-ae34-6033f265106d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:39.209766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-783672f4-d029-40f6-ae34-6033f265106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:39.277210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1264/2489] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:39.285756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.286054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.286360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.307191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:39.308435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:39.319480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1226/2490] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:39.325648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783672f4-d029-40f6-ae34-6033f265106d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.338532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783672f4-d029-40f6-ae34-6033f265106d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.344560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.345197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.345484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.345761 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.346483 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:19:39.346796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.348455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.361579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1265/2491] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:19:39.366863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.380556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866152-0059-4d8a-84cf-c887434fa521 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.385800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.386431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.386718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.387015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.387694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.389608 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.418679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1227/2492] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:39.423217 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.433688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f67b3-51e4-48d8-b02f-accfc6c4443c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.496577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.497218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.497555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.497767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.498439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.500428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.526496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1266/2493] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:39.528131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.538780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a43064-2e65-43b0-933f-4642e2c9a0d1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.544658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.545453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.545839 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.546460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.547307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.549036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.560815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1228/2494] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:39.565343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.575809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b50dc-9067-4bfc-8a46-2eca02217c39 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.582308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d706f12-67c1-496a-914f-44a1ec2383e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.582308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d706f12-67c1-496a-914f-44a1ec2383e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.582778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d706f12-67c1-496a-914f-44a1ec2383e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.582915 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d706f12-67c1-496a-914f-44a1ec2383e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.583827 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d706f12-67c1-496a-914f-44a1ec2383e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.585454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d706f12-67c1-496a-914f-44a1ec2383e0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.593216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:39.600258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1267/2495] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:39.605401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.615922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279437c-995a-41b1-8303-81a6e2bd1961 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.621250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.621927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.622213 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.622521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] PATH_INFO: `/v3/projects/71c0965a9b4749e7a35e798b784fd16e/users/3afd2d85f7554c90b2e3e8fe465ffcfb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.640142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c0965a9b4749e7a35e798b784fd16e, user_id=3afd2d85f7554c90b2e3e8fe465ffcfb, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.642865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.659701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1229/2496] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:39 2022] PUT /identity/v3/projects/71c0965a9b4749e7a35e798b784fd16e/users/3afd2d85f7554c90b2e3e8fe465ffcfb/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:39.671555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.671850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.672149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.707482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] MFA Rules not processed for user `3afd2d85f7554c90b2e3e8fe465ffcfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:39.709154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:39.781343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1268/2497] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:39.788054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.864121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834658a-e4b6-47fb-88dd-a945d95c73a2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.870910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.871776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.872274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.872556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.873242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.875345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:39.904611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1230/2498] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:39.908810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:39.920304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66986fc-0c01-4998-ad23-5ca8ffb4f2c0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:39.983247 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:39.983899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:39.984191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:39.984474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:39.985166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:39.987329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:40.014192 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1269/2499] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:39 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:40.015753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:40.026795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0936f-653e-4f43-bd72-ff183fc8f395 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:40.031498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:40.032114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:40.032532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:40.032770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:40.033396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:40.035114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:40.040543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:40.046532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1231/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:40.051025 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:40.061401 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed3ace-0c0b-4fb2-8bac-e9bb577c646e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:40.066514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:40.067215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:40.067507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:40.067766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] PATH_INFO: `/v3/projects/cfdb66c8fb2240fabd3bca44cd600eb7/users/c93cb9141372470aa233395e96b2e7bc/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:40.084980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdb66c8fb2240fabd3bca44cd600eb7, user_id=c93cb9141372470aa233395e96b2e7bc, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:40.087681 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:40.107201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1270/2501] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:40 2022] PUT /identity/v3/projects/cfdb66c8fb2240fabd3bca44cd600eb7/users/c93cb9141372470aa233395e96b2e7bc/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:40.113109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:40.124594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9aa11-ed24-42ac-9f30-6a4ae4fcb55a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:40.185824 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2185b8-07b7-4755-bf39-e1611b5d3b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:40.186449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2185b8-07b7-4755-bf39-e1611b5d3b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:40.186736 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2185b8-07b7-4755-bf39-e1611b5d3b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:40.187055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2185b8-07b7-4755-bf39-e1611b5d3b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:40.188317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2185b8-07b7-4755-bf39-e1611b5d3b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:40.189372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2185b8-07b7-4755-bf39-e1611b5d3b04 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:40.199802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1232/2502] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:40.204324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:40.219538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045db75d-1475-42d1-aaa3-4fa9dc896ce3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:40.225689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:40.226368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:40.226665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:40.226980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] PATH_INFO: `/v3/projects/cfdb66c8fb2240fabd3bca44cd600eb7/users/c93cb9141372470aa233395e96b2e7bc/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:40.229289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdb66c8fb2240fabd3bca44cd600eb7, user_id=c93cb9141372470aa233395e96b2e7bc, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:40.232003 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:40.243619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1271/2503] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:40 2022] PUT /identity/v3/projects/cfdb66c8fb2240fabd3bca44cd600eb7/users/c93cb9141372470aa233395e96b2e7bc/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:40.255528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:40.255967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:40.256371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:40.296833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] MFA Rules not processed for user `c93cb9141372470aa233395e96b2e7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:40.298074 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:40.372975 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1233/2504] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:40.386198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:40.388402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:40.441422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:40.462768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd1dff-6d5b-4533-bf53-6d82aeb824f0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:40.467837 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b474c-9b8d-49b8-94d8-1c08d4741220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:40.468624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b474c-9b8d-49b8-94d8-1c08d4741220 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:40.468992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b474c-9b8d-49b8-94d8-1c08d4741220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:40.469374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b474c-9b8d-49b8-94d8-1c08d4741220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:40.531959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b474c-9b8d-49b8-94d8-1c08d4741220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:40.533461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b474c-9b8d-49b8-94d8-1c08d4741220 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:40.543896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1272/2505] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:43.131019 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:43.197107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164ba72-fd23-44ef-bd9a-b4659c18f54a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:43.202035 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:43.202781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:43.203148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:43.203563 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] PATH_INFO: `/v3/users/3afd2d85f7554c90b2e3e8fe465ffcfb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:43.205057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3afd2d85f7554c90b2e3e8fe465ffcfb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:43.207117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:43.234769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:43.264653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3afd2d85f7554c90b2e3e8fe465ffcfb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.271367 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3afd2d85f7554c90b2e3e8fe465ffcfb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.272043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3afd2d85f7554c90b2e3e8fe465ffcfb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.284337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3afd2d85f7554c90b2e3e8fe465ffcfb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.291530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3afd2d85f7554c90b2e3e8fe465ffcfb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.297422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1234/2506] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:43 2022] DELETE /identity/v3/users/3afd2d85f7554c90b2e3e8fe465ffcfb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:43.302103 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:43.302578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:43.302696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:43.322954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:43.324205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:43.397846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1273/2507] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:43.411882 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:43.413440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:43.486614 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe014474-192e-402d-b244-5a084b2dd956 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:43.490582 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:43.491257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:43.491545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:43.491820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:43.501406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:43.502993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:43.513874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1235/2508] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:43.805265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:43.807145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:43.825729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:43.827706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0e1-6a02-4a51-a6d1-ccb172a35e38 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:43.828012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:43.833540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:43.834433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:43.834876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:43.835285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] PATH_INFO: `/v3/projects/71c0965a9b4749e7a35e798b784fd16e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:43.836596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c0965a9b4749e7a35e798b784fd16e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:43.838998 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:43.911758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c0965a9b4749e7a35e798b784fd16e', 'request_id': 'req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.919255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c0965a9b4749e7a35e798b784fd16e', 'request_id': 'req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.923913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fbd5f-ac48-4d34-b4ea-689989be535f service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:43.929013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c0965a9b4749e7a35e798b784fd16e', 'request_id': 'req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.942801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] The token cache is being invalidate because project 71c0965a9b4749e7a35e798b784fd16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:43.943252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.944036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:43.945929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1274/2509] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:43 2022] DELETE /identity/v3/projects/71c0965a9b4749e7a35e798b784fd16e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:43.950344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:43.951892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:43.970636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:43.980543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:43.998952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:43.999517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:43.999799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.000073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.001946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:44.014487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:44.016104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:44.026312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7817f6-37ed-4f0b-b74d-1d111cbcb8b8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:44.026506 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1236/2510] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:19:44.032323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:44.032490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:44.032782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.033093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] PATH_INFO: `/v3/users/c93cb9141372470aa233395e96b2e7bc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.034465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93cb9141372470aa233395e96b2e7bc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:44.036532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:44.093875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93cb9141372470aa233395e96b2e7bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.094560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93cb9141372470aa233395e96b2e7bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.095218 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93cb9141372470aa233395e96b2e7bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.102652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93cb9141372470aa233395e96b2e7bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.114930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-010abeb0-0e34-483d-9f7d-c33b947042be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93cb9141372470aa233395e96b2e7bc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.131301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1275/2511] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:43 2022] DELETE /identity/v3/users/c93cb9141372470aa233395e96b2e7bc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:19:44.585080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:44.596600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c64fb-955a-4508-81bf-f45358796fdc service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:44.663358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661b549-e407-4632-af55-818355c467fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:44.664121 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b549-e407-4632-af55-818355c467fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:44.664299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b549-e407-4632-af55-818355c467fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.664542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b549-e407-4632-af55-818355c467fd admin admin] PATH_INFO: `/v3/projects/cfdb66c8fb2240fabd3bca44cd600eb7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.665706 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661b549-e407-4632-af55-818355c467fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfdb66c8fb2240fabd3bca44cd600eb7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:44.667654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661b549-e407-4632-af55-818355c467fd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:44.722101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0661b549-e407-4632-af55-818355c467fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfdb66c8fb2240fabd3bca44cd600eb7', 'request_id': 'req-0661b549-e407-4632-af55-818355c467fd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.730509 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0661b549-e407-4632-af55-818355c467fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfdb66c8fb2240fabd3bca44cd600eb7', 'request_id': 'req-0661b549-e407-4632-af55-818355c467fd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.737148 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0661b549-e407-4632-af55-818355c467fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfdb66c8fb2240fabd3bca44cd600eb7', 'request_id': 'req-0661b549-e407-4632-af55-818355c467fd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.750856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0661b549-e407-4632-af55-818355c467fd admin admin] The token cache is being invalidate because project cfdb66c8fb2240fabd3bca44cd600eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:44.751178 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0661b549-e407-4632-af55-818355c467fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.751955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0661b549-e407-4632-af55-818355c467fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:44.753856 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1237/2512] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:44 2022] DELETE /identity/v3/projects/cfdb66c8fb2240fabd3bca44cd600eb7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:44.759448 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:44.759953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.760062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.780085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:44.781242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:44.843759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1276/2513] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:44.852989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:44.853343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.853623 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.875065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:44.876296 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:44.885959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1238/2514] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:44.890493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:44.900803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82156df-78a2-443c-8fa6-c6a91b97efc2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:44.905981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:44.906618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:44.906960 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.907243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.907953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:19:44.908233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:44.909950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:44.923041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1277/2515] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:19:44.927635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:44.937155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda0d6e-acd3-4bd5-92b5-53faf7b3b3a9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:44.942343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:44.942990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:44.943275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:44.943567 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:44.944223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:44.946110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:44.979055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1239/2516] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:44 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:44.983562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:44.994191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340dd2-a4fd-46a3-a8fe-76475bbb5056 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:45.022227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:45.060865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:45.061425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:45.061708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:45.061989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:45.062672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:45.064647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:45.087860 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1278/2517] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:44 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:45.092014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:45.102803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591178-b5dd-498a-b1f4-4f8ed971d377 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:45.108677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:45.109324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:45.109607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:45.109890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:45.110578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:45.112323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:45.122950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1240/2518] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:45.127304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:45.138529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2908d9-e528-492a-8261-a4ed58f40aab admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:45.144759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb450d-bfa3-4fb2-a385-7da954ca3957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:45.145389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb450d-bfa3-4fb2-a385-7da954ca3957 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:45.145656 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb450d-bfa3-4fb2-a385-7da954ca3957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:45.145944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb450d-bfa3-4fb2-a385-7da954ca3957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:45.146778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb450d-bfa3-4fb2-a385-7da954ca3957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:45.148427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb450d-bfa3-4fb2-a385-7da954ca3957 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:45.159452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1279/2519] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:45.163729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:45.173731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8f04f-8b10-4b8e-94fe-aa4f8a5dc046 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:45.179014 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:45.179729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:45.180015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:45.180297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] PATH_INFO: `/v3/projects/f5f6d69ec75146968efc2123a14ec34c/users/e79686b081be4357b41da798b89ba739/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:45.195735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f6d69ec75146968efc2123a14ec34c, user_id=e79686b081be4357b41da798b89ba739, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:45.198391 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:45.200601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:45.213354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1241/2520] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:45 2022] PUT /identity/v3/projects/f5f6d69ec75146968efc2123a14ec34c/users/e79686b081be4357b41da798b89ba739/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:45.224918 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:45.225436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:45.225947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:45.268037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] MFA Rules not processed for user `e79686b081be4357b41da798b89ba739`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:45.269280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:45.338591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1280/2521] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:45.349800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:45.351860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:45.419328 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485695b-de4c-47e9-b8ab-02280c5c42ef admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:45.423458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e31edd-2917-4fc6-957d-0162457de37a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:45.424089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e31edd-2917-4fc6-957d-0162457de37a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:45.424366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e31edd-2917-4fc6-957d-0162457de37a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:45.424631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e31edd-2917-4fc6-957d-0162457de37a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:45.433193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e31edd-2917-4fc6-957d-0162457de37a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:45.434632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e31edd-2917-4fc6-957d-0162457de37a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:45.443535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1242/2522] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:48.547582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:48.659136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:48.666170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21320-f8d8-4145-9301-8eb240ae5a36 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:48.671575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:48.672244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:48.672559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:48.672839 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] PATH_INFO: `/v3/users/e79686b081be4357b41da798b89ba739` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:48.674272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79686b081be4357b41da798b89ba739)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:48.676566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:48.726037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79686b081be4357b41da798b89ba739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:48.726697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79686b081be4357b41da798b89ba739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:48.727334 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79686b081be4357b41da798b89ba739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:48.734272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79686b081be4357b41da798b89ba739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:48.747536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79686b081be4357b41da798b89ba739'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:48.762112 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1281/2523] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:48 2022] DELETE /identity/v3/users/e79686b081be4357b41da798b89ba739 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:48.766655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:48.767181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:48.767546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:48.820974 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:48.822662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:48.887008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1243/2524] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:48.897559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:48.899833 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:49.028997 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ba739-3c6f-4af0-a038-0955e11ffd49 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.032914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd8bb6-6aad-43ae-b9f6-7a6776ef5da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.033588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd8bb6-6aad-43ae-b9f6-7a6776ef5da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.033906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd8bb6-6aad-43ae-b9f6-7a6776ef5da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.034188 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd8bb6-6aad-43ae-b9f6-7a6776ef5da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.043822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd8bb6-6aad-43ae-b9f6-7a6776ef5da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.045210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd8bb6-6aad-43ae-b9f6-7a6776ef5da2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.055152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1282/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:49.354989 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.356648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:49.372959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166676-6e53-4997-99b4-4127465b2b4f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.378707 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.379344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.379629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.380580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] PATH_INFO: `/v3/projects/f5f6d69ec75146968efc2123a14ec34c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.381820 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f6d69ec75146968efc2123a14ec34c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.383866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.434650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f6d69ec75146968efc2123a14ec34c', 'request_id': 'req-1a4325c7-1ffa-4f61-b879-b31447adc03a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:49.443470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f6d69ec75146968efc2123a14ec34c', 'request_id': 'req-1a4325c7-1ffa-4f61-b879-b31447adc03a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:49.450302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f6d69ec75146968efc2123a14ec34c', 'request_id': 'req-1a4325c7-1ffa-4f61-b879-b31447adc03a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:49.451649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:49.465786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] The token cache is being invalidate because project f5f6d69ec75146968efc2123a14ec34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:49.466078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:49.466795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1a4325c7-1ffa-4f61-b879-b31447adc03a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:49.468375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1244/2526] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:49 2022] DELETE /identity/v3/projects/f5f6d69ec75146968efc2123a14ec34c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:49.475495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.475783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.476066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.527387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:49.528598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:49.538601 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:49.590763 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1283/2527] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:49.599061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.599354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.599624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.623668 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:49.624855 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:49.634588 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1245/2528] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:49.639373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.650644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d165a4-6135-4f93-8354-1400d75bbcd4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.657196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.657841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.658128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.658409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.659162 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:19:49.659471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.665268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.678470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1284/2529] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:19:49.683380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.692985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20584fac-bead-437a-b8c9-5e5666de29c6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.698156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.698873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.699139 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.699977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.701120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.703428 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.759332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1246/2530] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:49 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:49.763940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.774862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc74f1-d914-4e67-91b8-582607dc3c25 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.835269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.835896 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.836224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.836499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.837236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.839214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.862685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1285/2531] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:49 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:49.866917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.877829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226485-693d-4f41-94fb-118dc8807084 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.883457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.884120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.884406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.884685 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.885395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.887269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.899138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1247/2532] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:19:49.905105 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.916982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0ecb-11b7-4af8-85aa-470d054f3865 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.922741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144db609-7073-46dd-8e71-4244ee601125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.923398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db609-7073-46dd-8e71-4244ee601125 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.923687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db609-7073-46dd-8e71-4244ee601125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.923966 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db609-7073-46dd-8e71-4244ee601125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.924680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db609-7073-46dd-8e71-4244ee601125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.926299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db609-7073-46dd-8e71-4244ee601125 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:49.937697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1286/2533] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:49.942719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:49.957803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b58bcd-46c0-43c5-b81c-f43a2887a137 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:49.966764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:49.967830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:49.968186 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:49.968559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] PATH_INFO: `/v3/projects/a79ecfe814ef4a4d9f80b4a6a87906b0/users/d4163bf003c949aead999caa845493ff/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:49.987198 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79ecfe814ef4a4d9f80b4a6a87906b0, user_id=d4163bf003c949aead999caa845493ff, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:49.990985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:50.013025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1248/2534] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:49 2022] PUT /identity/v3/projects/a79ecfe814ef4a4d9f80b4a6a87906b0/users/d4163bf003c949aead999caa845493ff/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:50.090431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:50.090727 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:50.091037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:50.119795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] MFA Rules not processed for user `d4163bf003c949aead999caa845493ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:50.120927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:50.186016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1287/2535] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:50.197866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:50.199952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:50.290467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9ef20-6679-47b4-bd26-7b1f7408f4b5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:50.295734 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431f8d7-5369-4689-a961-d69f423f874a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:50.296370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431f8d7-5369-4689-a961-d69f423f874a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:50.296657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431f8d7-5369-4689-a961-d69f423f874a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:50.296930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431f8d7-5369-4689-a961-d69f423f874a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:50.306279 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431f8d7-5369-4689-a961-d69f423f874a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:50.307781 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431f8d7-5369-4689-a961-d69f423f874a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:50.317109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1249/2536] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:52.913977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:52.927436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:52.980283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc10927-c7e6-415b-ba5f-18ed97ae80d3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:52.985238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:52.985895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:52.986205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:52.986452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] PATH_INFO: `/v3/users/d4163bf003c949aead999caa845493ff` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:52.988219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4163bf003c949aead999caa845493ff)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:52.990147 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:53.045252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4163bf003c949aead999caa845493ff'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.045798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4163bf003c949aead999caa845493ff'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.046411 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4163bf003c949aead999caa845493ff'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.054081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4163bf003c949aead999caa845493ff'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.067809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4163bf003c949aead999caa845493ff'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.085857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1288/2537] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:19:52 2022] DELETE /identity/v3/users/d4163bf003c949aead999caa845493ff => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:53.092304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.092813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.092813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.112804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:53.114022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:53.176701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1250/2538] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:53.190099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:53.191661 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:53.253130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e2fa0-cbb8-45a9-85d8-1ef3ddaa92f9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:53.257532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba3177-68fd-4684-82c6-eb3c7a61b83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:53.258157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba3177-68fd-4684-82c6-eb3c7a61b83c service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.258447 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba3177-68fd-4684-82c6-eb3c7a61b83c service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.258710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba3177-68fd-4684-82c6-eb3c7a61b83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.268224 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba3177-68fd-4684-82c6-eb3c7a61b83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:53.269679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba3177-68fd-4684-82c6-eb3c7a61b83c service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:53.279982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1289/2539] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:19:53.575743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:53.577217 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:53.585472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:53.589334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f44a21-2b6c-4829-b6d7-755162b46a87 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:53.595527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:53.596395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.596666 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.596950 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] PATH_INFO: `/v3/projects/a79ecfe814ef4a4d9f80b4a6a87906b0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.598218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79ecfe814ef4a4d9f80b4a6a87906b0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:53.600410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:53.660800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79ecfe814ef4a4d9f80b4a6a87906b0', 'request_id': 'req-225c5b33-6a92-4789-9f21-3b4514801dae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.669600 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79ecfe814ef4a4d9f80b4a6a87906b0', 'request_id': 'req-225c5b33-6a92-4789-9f21-3b4514801dae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.678062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79ecfe814ef4a4d9f80b4a6a87906b0', 'request_id': 'req-225c5b33-6a92-4789-9f21-3b4514801dae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.694374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] The token cache is being invalidate because project a79ecfe814ef4a4d9f80b4a6a87906b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:19:53.694670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.695435 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-225c5b33-6a92-4789-9f21-3b4514801dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:19:53.697057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1251/2540] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:19:53 2022] DELETE /identity/v3/projects/a79ecfe814ef4a4d9f80b4a6a87906b0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:53.702501 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.702800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.703119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.725081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:53.726298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:53.759463 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:53.792834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1290/2541] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:53.800751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.801046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.801326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.822183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:53.823358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:53.832382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1252/2542] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:53.836852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:53.847276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa00e82-94ca-48ac-b636-ab27d57db81f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:53.852280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:53.852982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.853391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.853723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.854519 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:19:53.854926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:53.856539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:53.868942 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1291/2543] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:19:53.873658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:53.883811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0156e-605d-42e2-b2bf-3308ab2a8c61 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:53.888616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:53.889246 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:53.889534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:53.889813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:53.890517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:53.892646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:53.930894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1253/2544] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:53.932495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:53.945953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ae831-1aa4-4407-93b0-29055f26d94f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:54.010086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:54.010728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.011051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.011332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.012015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:54.013944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:54.041780 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1292/2545] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:19:53 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:19:54.044415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:54.055804 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed880a24-e663-431d-a60f-5552a9f29bb3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:54.061802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:54.062863 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.063285 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.063646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.064595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:54.066483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:54.081168 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1254/2546] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:54.085241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:54.096111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425419-0142-4c8d-87fb-0f003f8c6cf6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:54.101189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9987765-a246-46b7-af74-684ee7363da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:54.101847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9987765-a246-46b7-af74-684ee7363da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.102125 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9987765-a246-46b7-af74-684ee7363da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.102404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9987765-a246-46b7-af74-684ee7363da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.103157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9987765-a246-46b7-af74-684ee7363da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:54.104961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9987765-a246-46b7-af74-684ee7363da3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:54.116272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1293/2547] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:19:54.120340 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:54.131301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928241d-4a57-4219-a610-f66dfa683254 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:54.136822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:54.137514 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.137812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.138068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] PATH_INFO: `/v3/projects/8840b41dd8364a8a976f04d4fe421b8e/users/82a7db7a0f544b3eb9487bf00adbb742/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.155955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8840b41dd8364a8a976f04d4fe421b8e, user_id=82a7db7a0f544b3eb9487bf00adbb742, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:54.158679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:54.192545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1255/2548] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:19:54 2022] PUT /identity/v3/projects/8840b41dd8364a8a976f04d4fe421b8e/users/82a7db7a0f544b3eb9487bf00adbb742/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:19:54.197936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0068c8-7cad-4b88-a5f7-de34b4f9014b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.198233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0068c8-7cad-4b88-a5f7-de34b4f9014b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.198516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0068c8-7cad-4b88-a5f7-de34b4f9014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.217676 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ff0068c8-7cad-4b88-a5f7-de34b4f9014b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:54.218925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ff0068c8-7cad-4b88-a5f7-de34b4f9014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:54.282567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1294/2549] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:54.293621 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:54.295739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:54.365587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f03fe-b67a-4d2a-8514-92d8f22d4d9a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:54.369694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:54.370304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.370587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.370862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.380497 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:54.381941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:54.391489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1256/2550] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:54.820674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ddba0-350c-4eb9-a25f-fda2363bbf66 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:54.821040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ddba0-350c-4eb9-a25f-fda2363bbf66 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:54.821413 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ddba0-350c-4eb9-a25f-fda2363bbf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:54.852089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-771ddba0-350c-4eb9-a25f-fda2363bbf66 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:54.853317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-771ddba0-350c-4eb9-a25f-fda2363bbf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:54.880396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:54.912453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1295/2551] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:19:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:54.925218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:54.926698 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:54.957109 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:19:55.001132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658211b9-4d38-4a2a-8173-b415e1329569 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:55.007049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:55.007686 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:55.007968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:55.008239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:55.018027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:55.019675 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:55.029796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1257/2552] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:19:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:19:55.045766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68714301-03e8-456c-8286-c56b890a95c4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:55.046348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68714301-03e8-456c-8286-c56b890a95c4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:55.046348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68714301-03e8-456c-8286-c56b890a95c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:55.077731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-68714301-03e8-456c-8286-c56b890a95c4 None None] MFA Rules not processed for user `82a7db7a0f544b3eb9487bf00adbb742`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:55.079011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-68714301-03e8-456c-8286-c56b890a95c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:55.213149 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1296/2553] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:19:55.225468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:19:55.226934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:55.315162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bc1c2-b83b-4636-a428-ff6d4e54dde9 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:19:55.326617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d3233-81f7-4beb-9192-d2a490e71db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:19:55.331056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3233-81f7-4beb-9192-d2a490e71db9 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:55.331056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3233-81f7-4beb-9192-d2a490e71db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:55.331056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3233-81f7-4beb-9192-d2a490e71db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:55.347895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d3233-81f7-4beb-9192-d2a490e71db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:19:55.350261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d3233-81f7-4beb-9192-d2a490e71db9 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:19:55.365229 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1258/2554] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:19:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:19:56.828529 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f34516-c480-46fa-86a3-0996343543fd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:19:56.829094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f34516-c480-46fa-86a3-0996343543fd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:19:56.829094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f34516-c480-46fa-86a3-0996343543fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:19:56.850164 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-08f34516-c480-46fa-86a3-0996343543fd None None] MFA Rules not processed for user `82a7db7a0f544b3eb9487bf00adbb742`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:19:56.851426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-08f34516-c480-46fa-86a3-0996343543fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:19:56.858864 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1297/2555] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:19:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:05.356717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=46 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:06.845015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=46 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:12.037329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1e648-0505-49e3-b9a6-ae88f2536f76 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:12.037847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1e648-0505-49e3-b9a6-ae88f2536f76 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:12.037957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1e648-0505-49e3-b9a6-ae88f2536f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:12.058896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cfa1e648-0505-49e3-b9a6-ae88f2536f76 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:12.060216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cfa1e648-0505-49e3-b9a6-ae88f2536f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:12.069310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1259/2556] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jul 28 01:20:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:12.095299 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f34516-c480-46fa-86a3-0996343543fd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:12.110788 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f34516-c480-46fa-86a3-0996343543fd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:12.118255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:12.118951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:12.119239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:12.119518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:12.132004 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:12.134281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:12.146813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1298/2557] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:20:22.053267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:22.141855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:27.958359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:27.958953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:27.959067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:27.982234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] MFA Rules not processed for user `8d5e9ee4b7f94c08bfb5eeb391bfa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:27.984164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:27.993627 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1260/2558] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jul 28 01:20:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:28.009371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:28.024710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72bf4-a32c-4117-ba83-e4bd4beceeb5 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:28.031237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4423ec5dc2f465086495db19afde175', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:28.031958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:28.032366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:28.032629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:28.041784 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:28.043425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:28.051990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1299/2559] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jul 28 01:20:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 01:20:32.549808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:32.551458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:32.564523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226766d-6b98-42ad-b18c-f14744e3db05 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:32.569234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:32.569913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:32.570193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:32.570515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:32.572517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:32.585740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:32.587223 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:32.597515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1261/2560] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:35.824987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:35.827153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:35.913605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b54f1-0291-4f7f-9aa2-6e8b5f70fe2d service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:35.919756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:35.920408 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:35.920689 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:35.920969 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:35.924445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:35.933956 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:35.935681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:35.940199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:35.941819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:35.951041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cce24-f23a-4aec-8b79-7a58064b2042 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:35.954036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1300/2561] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:35.957493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:35.958431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:35.959027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:35.959539 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] PATH_INFO: `/v3/users/c8ab1df6f15143de8f922a626317fa9c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:35.961336 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ab1df6f15143de8f922a626317fa9c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:35.964364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.026733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ab1df6f15143de8f922a626317fa9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.037032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ab1df6f15143de8f922a626317fa9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.037676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ab1df6f15143de8f922a626317fa9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.050304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ab1df6f15143de8f922a626317fa9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.058350 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1c5234da-f1cd-4f3d-99c5-a847cbf9f72e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ab1df6f15143de8f922a626317fa9c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.065128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1262/2562] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:20:35 2022] DELETE /identity/v3/users/c8ab1df6f15143de8f922a626317fa9c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:36.245468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.259717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b92cd-1db5-4fdd-bcdb-318fe990d655 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.323882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.324559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.325543 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.325930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] PATH_INFO: `/v3/projects/347bcdc7636e4f93af2597f853926b13` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.327298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] RBAC: Authorizing `identity:delete_project(project_id=347bcdc7636e4f93af2597f853926b13)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.329254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.380330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347bcdc7636e4f93af2597f853926b13', 'request_id': 'req-c3fcdf81-332b-48bb-b6b5-394457fcafea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.387436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347bcdc7636e4f93af2597f853926b13', 'request_id': 'req-c3fcdf81-332b-48bb-b6b5-394457fcafea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.396856 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347bcdc7636e4f93af2597f853926b13', 'request_id': 'req-c3fcdf81-332b-48bb-b6b5-394457fcafea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.407396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] The token cache is being invalidate because project 347bcdc7636e4f93af2597f853926b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:20:36.407660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.408344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c3fcdf81-332b-48bb-b6b5-394457fcafea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:36.409846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1301/2563] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:20:36 2022] DELETE /identity/v3/projects/347bcdc7636e4f93af2597f853926b13 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:36.415239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.415562 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.415849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.436462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:36.437664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:36.515987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1263/2564] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:36.527636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.528000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.528337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.554241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:36.562120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:36.565119 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:36.579815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1302/2565] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:20:36.586746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.603508 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d83b59-17d4-4a99-a27c-d27a3901541f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.611057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.612029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.612471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.612903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.614022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:20:36.614498 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.617292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.637957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1264/2566] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:20:36.643060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.653473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb2a97-55a1-4cf5-a506-8eb12457a20b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.659560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.660223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.660511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.660800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.661495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.663579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.695923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1303/2567] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:36 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:36.700206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.712011 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945838eb-cfab-4f94-93c9-3dce8ebb5807 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.773843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:36.797434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.798069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.798357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.798636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.799399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.801341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.825410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1265/2568] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:36 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:36.829666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.845877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc06f8b-589d-490c-b874-32da403ad22f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.853867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.854821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.855297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.855688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.856706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.859184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.872193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1304/2569] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:36.876676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.890343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6788fe-5133-4ccb-8a22-e11c08f8e43c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.898907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfe1e8-0be0-4404-a6eb-cc03fa93f31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.898907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfe1e8-0be0-4404-a6eb-cc03fa93f31c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.898907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfe1e8-0be0-4404-a6eb-cc03fa93f31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.898907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfe1e8-0be0-4404-a6eb-cc03fa93f31c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.899387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfe1e8-0be0-4404-a6eb-cc03fa93f31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.900263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfe1e8-0be0-4404-a6eb-cc03fa93f31c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.912241 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1266/2570] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:36.917361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:36.929908 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5473-f2a7-4d81-a950-d58653dccef3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:36.936734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:36.937450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.937733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.938015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] PATH_INFO: `/v3/projects/d7a91b709d6b44fb9759898679a53ac6/users/7eb797f227d949f092f7328f5b0f0ee7/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:36.954100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a91b709d6b44fb9759898679a53ac6, user_id=7eb797f227d949f092f7328f5b0f0ee7, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:36.956739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:36.973085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1305/2571] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:36 2022] PUT /identity/v3/projects/d7a91b709d6b44fb9759898679a53ac6/users/7eb797f227d949f092f7328f5b0f0ee7/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:36.984805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:36.985373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:36.985373 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:37.015967 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] MFA Rules not processed for user `7eb797f227d949f092f7328f5b0f0ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:37.017226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:37.095892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1267/2572] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:37.107810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:37.109597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:37.187668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430a152-2909-47ea-89f7-2fc81ad0e617 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:37.195685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2cb49-da95-4558-a16e-4d37906b36cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:37.196489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2cb49-da95-4558-a16e-4d37906b36cc service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:37.196787 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2cb49-da95-4558-a16e-4d37906b36cc service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:37.197071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2cb49-da95-4558-a16e-4d37906b36cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:37.206369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2cb49-da95-4558-a16e-4d37906b36cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:37.207934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2cb49-da95-4558-a16e-4d37906b36cc service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:37.217393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1306/2573] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:37.441277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:37.508091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fc286-372d-47b0-bfc6-e6ebb8dea9a2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:37.513425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:37.514067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:37.514343 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:37.514626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] PATH_INFO: `/v3/users/7eb797f227d949f092f7328f5b0f0ee7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:37.515879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb797f227d949f092f7328f5b0f0ee7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:37.517802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:37.567236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb797f227d949f092f7328f5b0f0ee7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:37.569740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:37.575910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb797f227d949f092f7328f5b0f0ee7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:37.576951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb797f227d949f092f7328f5b0f0ee7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:37.590299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb797f227d949f092f7328f5b0f0ee7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:37.597353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb797f227d949f092f7328f5b0f0ee7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:37.603335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1268/2574] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:20:37 2022] DELETE /identity/v3/users/7eb797f227d949f092f7328f5b0f0ee7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:37.607973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:37.608390 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:37.608715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:37.629504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:37.631145 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:37.693504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1307/2575] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:37.703464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:37.705184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:37.763719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cecd7-bda4-4733-a413-27243aa3017f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:37.767663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f921e5-873a-4e39-a468-19541fc6703a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:37.768293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f921e5-873a-4e39-a468-19541fc6703a service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:37.768570 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f921e5-873a-4e39-a468-19541fc6703a service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:37.768846 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f921e5-873a-4e39-a468-19541fc6703a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:37.779104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f921e5-873a-4e39-a468-19541fc6703a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:37.780542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f921e5-873a-4e39-a468-19541fc6703a service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:37.791759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1269/2576] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:38.079720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.081344 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:38.095071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19bae-cba9-4fc0-bdde-8ae5354b8a46 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.101158 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.101890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.102338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.102626 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] PATH_INFO: `/v3/projects/d7a91b709d6b44fb9759898679a53ac6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.103983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a91b709d6b44fb9759898679a53ac6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.106001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.134440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:38.161636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a91b709d6b44fb9759898679a53ac6', 'request_id': 'req-39ebe007-ad82-42f0-a96d-777601f254cc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:38.168237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a91b709d6b44fb9759898679a53ac6', 'request_id': 'req-39ebe007-ad82-42f0-a96d-777601f254cc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:38.177981 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a91b709d6b44fb9759898679a53ac6', 'request_id': 'req-39ebe007-ad82-42f0-a96d-777601f254cc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:38.189949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] The token cache is being invalidate because project d7a91b709d6b44fb9759898679a53ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:20:38.190340 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:38.191114 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-39ebe007-ad82-42f0-a96d-777601f254cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:38.192765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1308/2577] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:20:38 2022] DELETE /identity/v3/projects/d7a91b709d6b44fb9759898679a53ac6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:38.198228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.198719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.198827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.221458 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:38.222771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:38.292022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1270/2578] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:38.300017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.300392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.300677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.322943 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:38.324176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:38.332581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1309/2579] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:38.337180 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.347156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1ad3-7775-4f5d-adab-b047ce5fa070 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.352833 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.353474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.353749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.354025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.354739 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:20:38.355069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.356763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.379236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1271/2580] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:20:38.384975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.400284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492148-4db7-4142-bc1c-4aad55be9703 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.408409 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.409057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.409403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.409801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.410478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.412731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.447846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1310/2581] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:38.449490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.460066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207109f2-3f28-4faa-9d31-7d58e1afc45a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.511705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:38.519434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.520071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.520362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.520663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.521334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.523727 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.552376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1272/2582] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:38.553665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.565020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f734259-f28f-4035-abbb-aa7cbb9458c9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.570279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.570947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.571241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.571645 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.572197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.573807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.585063 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1311/2583] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:20:38.589667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.603003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23b10-08e0-4b54-b776-ef450b6be5aa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.608835 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bed33-cfc2-4877-8ec7-8457a896cb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.609472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bed33-cfc2-4877-8ec7-8457a896cb5f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.609752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bed33-cfc2-4877-8ec7-8457a896cb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.610024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bed33-cfc2-4877-8ec7-8457a896cb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.610756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bed33-cfc2-4877-8ec7-8457a896cb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.612412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bed33-cfc2-4877-8ec7-8457a896cb5f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.624667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1273/2584] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:38.629084 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.640766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ecec8-78e2-4675-b41e-58854b806dd6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.645991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.646688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.647012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.647290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] PATH_INFO: `/v3/projects/28d4e7d4b34c477493e3a0da327552a4/users/b7e7dc0d6e90413f9cd6b899957ac398/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.663071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d4e7d4b34c477493e3a0da327552a4, user_id=b7e7dc0d6e90413f9cd6b899957ac398, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.665796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.678323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1312/2585] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:38 2022] PUT /identity/v3/projects/28d4e7d4b34c477493e3a0da327552a4/users/b7e7dc0d6e90413f9cd6b899957ac398/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:38.688867 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.689162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.689468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.718449 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] MFA Rules not processed for user `b7e7dc0d6e90413f9cd6b899957ac398`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:38.719722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:38.786894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1274/2586] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:20:38.791523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.864879 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdff45-de04-487f-8cb0-a74dd4ac0cfe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.870300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.870976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.871239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.871541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.872211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.874172 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:38.907243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1313/2587] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:38.908619 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:38.922044 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364d855-ed6e-4b87-95fd-77695bcf1e5c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:38.981172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:38.981803 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:38.982087 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:38.982360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:38.983090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:38.985075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:39.011137 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1275/2588] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:38 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:20:39.015537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:39.027297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957be98c-d8f3-4bde-bd0c-17ea836ceffe admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:39.033044 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:39.033678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:39.033965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:39.034239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:39.034972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:39.036569 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:39.048517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1314/2589] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:39.053207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:39.064207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc4648-a849-4fd4-b67d-e0916a3ff9b8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:39.070761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:39.071482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:39.071773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:39.072061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] PATH_INFO: `/v3/projects/25fb63a8595f44bf99c18619f6fdf47f/users/e4299235ca0b43bfa657a886b0dca8aa/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:39.088593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fb63a8595f44bf99c18619f6fdf47f, user_id=e4299235ca0b43bfa657a886b0dca8aa, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:39.091364 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:39.106673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1276/2590] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:39 2022] PUT /identity/v3/projects/25fb63a8595f44bf99c18619f6fdf47f/users/e4299235ca0b43bfa657a886b0dca8aa/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:39.110961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:39.121862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b474f-1322-4871-b331-f0727e84dc42 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:39.140891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:39.182632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade01e2-daad-4f4d-8eae-171202ca7622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:39.183302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade01e2-daad-4f4d-8eae-171202ca7622 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:39.183593 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade01e2-daad-4f4d-8eae-171202ca7622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:39.183871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade01e2-daad-4f4d-8eae-171202ca7622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:39.184583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade01e2-daad-4f4d-8eae-171202ca7622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:39.186191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade01e2-daad-4f4d-8eae-171202ca7622 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:39.196884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1315/2591] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:39.201424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:39.212464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd214f-4c4c-48d8-9c01-8a18848def7c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:39.217812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:39.218495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:39.218780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:39.219111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] PATH_INFO: `/v3/projects/25fb63a8595f44bf99c18619f6fdf47f/users/e4299235ca0b43bfa657a886b0dca8aa/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:39.221159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fb63a8595f44bf99c18619f6fdf47f, user_id=e4299235ca0b43bfa657a886b0dca8aa, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:39.223891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:39.234743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1277/2592] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:39 2022] PUT /identity/v3/projects/25fb63a8595f44bf99c18619f6fdf47f/users/e4299235ca0b43bfa657a886b0dca8aa/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:39.246540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:39.246867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:39.247152 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:39.277081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] MFA Rules not processed for user `e4299235ca0b43bfa657a886b0dca8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:39.278374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:39.361244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1316/2593] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:20:39.373239 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:39.375117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:39.402106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:39.454826 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc482bd-b77f-45f1-a401-9722cd278229 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:39.458618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a6eca-7074-47fe-b250-76a5433c848b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:39.459304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a6eca-7074-47fe-b250-76a5433c848b service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:39.459907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a6eca-7074-47fe-b250-76a5433c848b service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:39.460128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a6eca-7074-47fe-b250-76a5433c848b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:39.480326 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a6eca-7074-47fe-b250-76a5433c848b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:39.482237 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a6eca-7074-47fe-b250-76a5433c848b service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:39.494222 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1278/2594] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:39.941220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:40.015016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482b81-b78e-4129-81f0-4aca236183eb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:40.020000 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:40.020660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:40.020948 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:40.021227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] PATH_INFO: `/v3/users/b7e7dc0d6e90413f9cd6b899957ac398` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:40.022428 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e7dc0d6e90413f9cd6b899957ac398)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:40.024460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:40.076952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:40.079470 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e7dc0d6e90413f9cd6b899957ac398'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.080565 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e7dc0d6e90413f9cd6b899957ac398'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.081715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e7dc0d6e90413f9cd6b899957ac398'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.091602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e7dc0d6e90413f9cd6b899957ac398'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.104679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e7dc0d6e90413f9cd6b899957ac398'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.117672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1317/2595] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:20:39 2022] DELETE /identity/v3/users/b7e7dc0d6e90413f9cd6b899957ac398 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:40.122427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc673b-cee7-4221-9093-937e77252697 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:40.122664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc673b-cee7-4221-9093-937e77252697 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:40.122952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc673b-cee7-4221-9093-937e77252697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:40.145884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-52cc673b-cee7-4221-9093-937e77252697 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:40.147323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52cc673b-cee7-4221-9093-937e77252697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:40.215920 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1279/2596] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:40.226068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:40.227921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:40.291559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d84e1-4008-4d90-a6e7-4579bbd4d05c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:40.296580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:40.297427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:40.297736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:40.298021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:40.308261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:40.309710 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:40.319554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1318/2597] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:40.611894 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc673b-cee7-4221-9093-937e77252697 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:40.613371 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52cc673b-cee7-4221-9093-937e77252697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:40.627517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc673b-cee7-4221-9093-937e77252697 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:40.633841 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:40.634402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:40.635308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:40.635584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] PATH_INFO: `/v3/projects/28d4e7d4b34c477493e3a0da327552a4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:40.637027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d4e7d4b34c477493e3a0da327552a4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:40.639027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:40.692033 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d4e7d4b34c477493e3a0da327552a4', 'request_id': 'req-1825349c-3d29-476f-9c03-c608ad25ea93'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.704303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d4e7d4b34c477493e3a0da327552a4', 'request_id': 'req-1825349c-3d29-476f-9c03-c608ad25ea93'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.710814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d4e7d4b34c477493e3a0da327552a4', 'request_id': 'req-1825349c-3d29-476f-9c03-c608ad25ea93'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.723231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] The token cache is being invalidate because project 28d4e7d4b34c477493e3a0da327552a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:20:40.723521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.724184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.725722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1280/2598] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:20:40 2022] DELETE /identity/v3/projects/28d4e7d4b34c477493e3a0da327552a4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:40.729954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:40.739987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ca86f-4634-4292-b2df-fcd7afeeb045 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:40.799811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:40.800566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:40.800919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:40.801296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] PATH_INFO: `/v3/users/e4299235ca0b43bfa657a886b0dca8aa` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:40.802547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4299235ca0b43bfa657a886b0dca8aa)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:40.804744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:40.851774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4299235ca0b43bfa657a886b0dca8aa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.852442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4299235ca0b43bfa657a886b0dca8aa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.853097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4299235ca0b43bfa657a886b0dca8aa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.860068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4299235ca0b43bfa657a886b0dca8aa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.869629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:40.879364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bd78c98a-3e01-46d9-bff7-ebffdd5b9098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4299235ca0b43bfa657a886b0dca8aa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:40.895248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1319/2599] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:20:40 2022] DELETE /identity/v3/users/e4299235ca0b43bfa657a886b0dca8aa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:41.220763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.222418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:41.237847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825349c-3d29-476f-9c03-c608ad25ea93 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.283516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:41.303018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.303656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.303942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.304324 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] PATH_INFO: `/v3/projects/25fb63a8595f44bf99c18619f6fdf47f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.305504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25fb63a8595f44bf99c18619f6fdf47f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.307469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.370793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25fb63a8595f44bf99c18619f6fdf47f', 'request_id': 'req-abddccfc-7e2d-4587-887c-78465c4aede1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:41.381470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25fb63a8595f44bf99c18619f6fdf47f', 'request_id': 'req-abddccfc-7e2d-4587-887c-78465c4aede1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:41.389998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25fb63a8595f44bf99c18619f6fdf47f', 'request_id': 'req-abddccfc-7e2d-4587-887c-78465c4aede1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:41.403267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] The token cache is being invalidate because project 25fb63a8595f44bf99c18619f6fdf47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:20:41.403560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:41.404247 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-abddccfc-7e2d-4587-887c-78465c4aede1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:41.405814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1281/2600] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:20:41 2022] DELETE /identity/v3/projects/25fb63a8595f44bf99c18619f6fdf47f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:41.411043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.411540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.411655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.433551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:41.434802 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:41.499723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1320/2601] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:41.508291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.508578 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.508859 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.529496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:41.530730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:41.539251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1282/2602] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:41.544328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.558874 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ac20-684a-400b-891b-fc27d1d8a559 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.564514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.565141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.565424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.565734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.566440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:20:41.566751 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.568392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.580944 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1321/2603] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:20:41.585632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.595649 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679bb40-503f-4d21-bcd3-dd7e265f391d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.601208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.601801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.602086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.602361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.603090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.605126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.636897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1283/2604] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:41 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:41.642091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.657962 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745e4e-2e18-48bd-bd95-493d279dfcc5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.722876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.723156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.723244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.723518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.724199 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.726210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.748393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1322/2605] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:41 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:41.752297 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.763440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3616f7-09ee-42f4-b9d0-dae00529b4ab admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.768330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.768992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.769280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.769559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.770275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.771929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.782352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1284/2606] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:41.786692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.797118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c2be0-33fc-43ea-b643-c6c023483ce4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.802100 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e481f-7628-4420-83b6-141a61300f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.802743 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e481f-7628-4420-83b6-141a61300f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.803062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e481f-7628-4420-83b6-141a61300f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.803342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e481f-7628-4420-83b6-141a61300f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.804049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e481f-7628-4420-83b6-141a61300f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.805697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e481f-7628-4420-83b6-141a61300f35 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.817530 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1323/2607] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:41.821990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:41.832174 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63ed3-eff4-4700-8b6e-1ac4f7b0954c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:41.837093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:41.837808 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.838097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.838380 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] PATH_INFO: `/v3/projects/f9a209466a4049a39d163cbe2ec2ac4f/users/fa486d83de8c42728abb3d139697e701/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.857269 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a209466a4049a39d163cbe2ec2ac4f, user_id=fa486d83de8c42728abb3d139697e701, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:41.861368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:41.880586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1285/2608] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:41 2022] PUT /identity/v3/projects/f9a209466a4049a39d163cbe2ec2ac4f/users/fa486d83de8c42728abb3d139697e701/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:41.891809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:41.892122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:41.892398 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:41.923615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] MFA Rules not processed for user `fa486d83de8c42728abb3d139697e701`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:41.924937 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:41.978995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:42.008081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1324/2609] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:42.012870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.077467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca5971-cc57-42f5-bc5a-ed1e8f815422 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.082257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.082921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.083205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.083475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.084144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.086126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.101910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:42.116551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1286/2610] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:42 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:42.121362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.133207 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04f19-3554-41ad-ba6f-b1302a90d5d3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.193399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.194043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.194323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.194611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.195363 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.197285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.225442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1325/2611] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:42 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:20:42.231111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.242020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d005d-ff4b-4276-8e4f-4a5278b5ceec admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.247990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.248611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.248995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.249275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.249980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.251655 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.263199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1287/2612] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:42.267443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.277933 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80510-b53f-4cc0-8c49-d38cb1160925 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.282749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.283476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.283760 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.284068 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] PATH_INFO: `/v3/projects/e3a08352c5fe469bb5874786ad492dce/users/394c97b9c5bb443a96b488ba3bbc62eb/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.300772 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a08352c5fe469bb5874786ad492dce, user_id=394c97b9c5bb443a96b488ba3bbc62eb, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.303502 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.317437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1326/2613] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:42 2022] PUT /identity/v3/projects/e3a08352c5fe469bb5874786ad492dce/users/394c97b9c5bb443a96b488ba3bbc62eb/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:20:42.321746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.331802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554da70-783e-46ba-b981-077458a025f0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.391242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0baa-50b6-4698-bc3d-169b2a224fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.391875 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0baa-50b6-4698-bc3d-169b2a224fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.392160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0baa-50b6-4698-bc3d-169b2a224fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.392434 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0baa-50b6-4698-bc3d-169b2a224fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.393131 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0baa-50b6-4698-bc3d-169b2a224fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.394754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0baa-50b6-4698-bc3d-169b2a224fa7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.405018 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1288/2614] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:42.410975 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.422191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec32a1-b80f-4d40-8c42-82d728b5bf24 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.427752 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.428432 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.428716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.429001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] PATH_INFO: `/v3/projects/e3a08352c5fe469bb5874786ad492dce/users/394c97b9c5bb443a96b488ba3bbc62eb/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.431287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a08352c5fe469bb5874786ad492dce, user_id=394c97b9c5bb443a96b488ba3bbc62eb, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.433935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.447443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1327/2615] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:42 2022] PUT /identity/v3/projects/e3a08352c5fe469bb5874786ad492dce/users/394c97b9c5bb443a96b488ba3bbc62eb/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:42.462094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.462361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.462651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.494226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] MFA Rules not processed for user `394c97b9c5bb443a96b488ba3bbc62eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:42.495502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:42.567094 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1289/2616] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:42.578279 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:42.580768 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:42.652545 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efee6a-aed9-4370-9e12-fd3c04ab328d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:42.657010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf5d9b-4289-4f12-9e03-4c6fd5ef41ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:42.657682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf5d9b-4289-4f12-9e03-4c6fd5ef41ea service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:42.657968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf5d9b-4289-4f12-9e03-4c6fd5ef41ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:42.658251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf5d9b-4289-4f12-9e03-4c6fd5ef41ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:42.668832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf5d9b-4289-4f12-9e03-4c6fd5ef41ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:42.670294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf5d9b-4289-4f12-9e03-4c6fd5ef41ea service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:42.679961 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1328/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:43.303693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:43.365933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c00e-7662-4d04-a371-beed837e2666 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:43.370704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:43.371386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:43.371672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:43.371948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] PATH_INFO: `/v3/users/fa486d83de8c42728abb3d139697e701` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:43.373138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa486d83de8c42728abb3d139697e701)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:43.375104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:43.380880 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:43.432679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa486d83de8c42728abb3d139697e701'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:43.441584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa486d83de8c42728abb3d139697e701'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:43.442211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa486d83de8c42728abb3d139697e701'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:43.455080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa486d83de8c42728abb3d139697e701'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:43.462818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa486d83de8c42728abb3d139697e701'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:43.469104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1290/2618] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:20:43 2022] DELETE /identity/v3/users/fa486d83de8c42728abb3d139697e701 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:43.473909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:43.474394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:43.474518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:43.494091 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:43.495354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:43.529834 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:43.562906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1329/2619] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:43.573521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:43.575124 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:43.640114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a247cb8-c7d2-4add-bbf0-d1828264d76d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:43.644128 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:43.644762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:43.645048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:43.645323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:43.655251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:43.656713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:43.667151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1291/2620] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:43.971919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:43.973249 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:43.994533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f3294-79d9-4325-912a-190d0a32ae53 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:44.000538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:44.001170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.001465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.002303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] PATH_INFO: `/v3/projects/f9a209466a4049a39d163cbe2ec2ac4f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.003728 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a209466a4049a39d163cbe2ec2ac4f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:44.006289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:44.063215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a209466a4049a39d163cbe2ec2ac4f', 'request_id': 'req-33691dd1-4c04-4cbd-b34e-98e108d132c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.072561 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a209466a4049a39d163cbe2ec2ac4f', 'request_id': 'req-33691dd1-4c04-4cbd-b34e-98e108d132c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.087184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a209466a4049a39d163cbe2ec2ac4f', 'request_id': 'req-33691dd1-4c04-4cbd-b34e-98e108d132c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.099220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] The token cache is being invalidate because project f9a209466a4049a39d163cbe2ec2ac4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:20:44.099516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.100304 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.101946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1330/2621] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:20:43 2022] DELETE /identity/v3/projects/f9a209466a4049a39d163cbe2ec2ac4f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:44.106257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:44.116480 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790af31f-c8a9-4328-bc74-0308bcd46683 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:44.184883 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:44.185469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.185760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.186041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] PATH_INFO: `/v3/users/394c97b9c5bb443a96b488ba3bbc62eb` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.187299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=394c97b9c5bb443a96b488ba3bbc62eb)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:44.189240 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:44.229302 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:44.249554 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394c97b9c5bb443a96b488ba3bbc62eb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.261251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394c97b9c5bb443a96b488ba3bbc62eb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.261942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394c97b9c5bb443a96b488ba3bbc62eb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.276310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394c97b9c5bb443a96b488ba3bbc62eb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.283780 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c1cbd143-6611-47fa-8855-a3ce1d0d3a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394c97b9c5bb443a96b488ba3bbc62eb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.289915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1292/2622] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:20:44 2022] DELETE /identity/v3/users/394c97b9c5bb443a96b488ba3bbc62eb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:44.625782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:44.627404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:44.644219 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33691dd1-4c04-4cbd-b34e-98e108d132c3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:44.705323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:44.705972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.706253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.706525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] PATH_INFO: `/v3/projects/e3a08352c5fe469bb5874786ad492dce` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.707749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a08352c5fe469bb5874786ad492dce)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:44.709672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:44.766454 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a08352c5fe469bb5874786ad492dce', 'request_id': 'req-e1a02475-2c61-41d8-95d1-a61efa64f512'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.772794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a08352c5fe469bb5874786ad492dce', 'request_id': 'req-e1a02475-2c61-41d8-95d1-a61efa64f512'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.782124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a08352c5fe469bb5874786ad492dce', 'request_id': 'req-e1a02475-2c61-41d8-95d1-a61efa64f512'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.783559 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:44.795129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] The token cache is being invalidate because project e3a08352c5fe469bb5874786ad492dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:20:44.795425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.796074 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1a02475-2c61-41d8-95d1-a61efa64f512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:20:44.797635 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1331/2623] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:20:44 2022] DELETE /identity/v3/projects/e3a08352c5fe469bb5874786ad492dce => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:44.802724 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.803258 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.803361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.823900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:44.826306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:44.897744 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1293/2624] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:44.906539 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.906847 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.907143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.928195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:44.929410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:44.939104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1332/2625] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:44.943719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:44.954496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bbbee-e303-4b42-b542-2230312b1003 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:44.960185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:44.960806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.961085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.961358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.962112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:20:44.962425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:44.964089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:44.976189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1294/2626] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:20:44.980697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:44.990143 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf8374-8538-4ee3-8b53-eac350bde46d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:44.995746 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:44.996366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:44.996653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:44.996935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:44.997630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:44.999627 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:45.031142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1333/2627] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:44 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:20:45.035346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:45.045584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2decf7-c489-4e49-956d-d33c713a0d2b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:45.103242 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946d855-4841-4003-a911-171637463f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:45.103912 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946d855-4841-4003-a911-171637463f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:45.104192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946d855-4841-4003-a911-171637463f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:45.104468 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946d855-4841-4003-a911-171637463f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:45.105142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946d855-4841-4003-a911-171637463f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:45.107107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946d855-4841-4003-a911-171637463f30 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:45.130424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1295/2628] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:45 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:45.134714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:45.145597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cb54a-4a0a-448a-bfe9-b5710bed2f19 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:45.150589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:45.151251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:45.151534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:45.151809 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:45.152541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:45.154130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:45.164789 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1334/2629] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:45.168827 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946d855-4841-4003-a911-171637463f30 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:45.179489 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946d855-4841-4003-a911-171637463f30 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:45.184633 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03432d-08ca-42ed-bddc-8a15cf4487f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:45.185295 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03432d-08ca-42ed-bddc-8a15cf4487f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:45.185584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03432d-08ca-42ed-bddc-8a15cf4487f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:45.185864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03432d-08ca-42ed-bddc-8a15cf4487f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:45.186580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03432d-08ca-42ed-bddc-8a15cf4487f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:45.188231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03432d-08ca-42ed-bddc-8a15cf4487f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:45.199671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1296/2630] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:45.204577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:45.220342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c942a-00bc-405b-a0ff-6d28d74c221f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:45.228332 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:45.228970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:45.229252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:45.229526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] PATH_INFO: `/v3/projects/f9d1dce0e819423b97d775e29216df21/users/3bfaad5e15074146b5bf7e51db52ccba/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:45.244646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d1dce0e819423b97d775e29216df21, user_id=3bfaad5e15074146b5bf7e51db52ccba, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:45.247336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:45.263269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1335/2631] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:45 2022] PUT /identity/v3/projects/f9d1dce0e819423b97d775e29216df21/users/3bfaad5e15074146b5bf7e51db52ccba/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:45.271629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81435a-2603-4247-80c0-f183a9e8d2a8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:45.271924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81435a-2603-4247-80c0-f183a9e8d2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:45.272334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81435a-2603-4247-80c0-f183a9e8d2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:45.299617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6d81435a-2603-4247-80c0-f183a9e8d2a8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:45.300795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6d81435a-2603-4247-80c0-f183a9e8d2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:45.309901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:45.364197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1297/2632] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:45.373952 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:45.376010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:45.437554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ea60d-b6a7-46c7-8811-37d27a341bcf admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:45.442252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:45.443050 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:45.443336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:45.443609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:45.455027 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:45.456471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:45.466234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1336/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:46.212107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.212657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.212657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.244645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] MFA Rules not processed for user `3bfaad5e15074146b5bf7e51db52ccba`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:46.245871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:46.313056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1298/2634] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:46.318034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:46.328093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56220ff4-7b2f-4737-8b4f-97225a84ebb4 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:46.334071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:46.334709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.335021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.335312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.336033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:46.337980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:46.368016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1337/2635] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:20:46 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:46.372218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:46.386227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb503c-712b-4815-9f2b-c98ae23eabac None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:46.448921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:46.449565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.449847 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.450123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.450809 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:46.452763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:46.479689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1299/2636] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:20:46 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:20:46.484797 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:46.499437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b8474-3187-4c50-a406-4a6ce1326d98 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:46.505606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:46.506237 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.506517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.506766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.507512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:46.509104 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:46.520301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1338/2637] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:20:46.524898 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:46.536930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e927-399f-49d0-8b71-a8cf4a233f99 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:46.542886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:46.543594 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.543872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.544144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] PATH_INFO: `/v3/projects/dad16612be6441a88785c89c692f3ed1/users/1b26073d75034f22aacc9935ceca2fc1/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.560244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad16612be6441a88785c89c692f3ed1, user_id=1b26073d75034f22aacc9935ceca2fc1, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:46.563045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:46.578335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1300/2638] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:46 2022] PUT /identity/v3/projects/dad16612be6441a88785c89c692f3ed1/users/1b26073d75034f22aacc9935ceca2fc1/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:46.583189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:46.595904 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d947ae-1225-4d55-9797-b0d301274b44 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:46.651852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:46.660185 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a5966-ec20-4f3b-88f4-73541178255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:46.660857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a5966-ec20-4f3b-88f4-73541178255d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.661153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a5966-ec20-4f3b-88f4-73541178255d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.661439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a5966-ec20-4f3b-88f4-73541178255d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.662151 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a5966-ec20-4f3b-88f4-73541178255d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:46.664541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a5966-ec20-4f3b-88f4-73541178255d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:46.680156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1339/2639] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:20:46.685152 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:46.696862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c272ad-dccd-4ed9-b507-c9fb68c6eaeb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:46.703059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:46.703709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:46.704006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:46.704280 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] PATH_INFO: `/v3/projects/dad16612be6441a88785c89c692f3ed1/users/1b26073d75034f22aacc9935ceca2fc1/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:46.706517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad16612be6441a88785c89c692f3ed1, user_id=1b26073d75034f22aacc9935ceca2fc1, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:46.709192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:46.712579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:46.721957 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1301/2640] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:20:46 2022] PUT /identity/v3/projects/dad16612be6441a88785c89c692f3ed1/users/1b26073d75034f22aacc9935ceca2fc1/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:20:47.372201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:47.372729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:47.372830 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:47.403387 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] MFA Rules not processed for user `1b26073d75034f22aacc9935ceca2fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:47.404965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:47.483113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1340/2641] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:20:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:47.497587 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:47.499911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:47.570191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f8f69-ff28-463f-a6f2-d470e87440bb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:47.574298 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:47.575083 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:47.575372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:47.575647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:47.637629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:47.639259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:47.650962 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1302/2642] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:49.036397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:49.047393 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c6664-92ae-41b1-af5a-aa44a45a4964 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:49.051497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c22f1d-6c63-468d-9cf9-694a405da703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:49.052130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c22f1d-6c63-468d-9cf9-694a405da703 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:49.052420 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c22f1d-6c63-468d-9cf9-694a405da703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:49.052703 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c22f1d-6c63-468d-9cf9-694a405da703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:49.062238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c22f1d-6c63-468d-9cf9-694a405da703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:49.063726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c22f1d-6c63-468d-9cf9-694a405da703 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:49.074129 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1341/2643] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:55.100045 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:55.101536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:55.179670 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9aeaa-c4cb-4733-8770-4749f248bfa3 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:55.186205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:55.186865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:55.187160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:55.187440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:55.189333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:55.203073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:55.204921 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f798d-c1d0-4f0c-b3e6-990853e02519 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:55.216221 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1303/2644] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:57.959990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f163978-01d1-461e-a658-4880b3ea174d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:57.960644 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f163978-01d1-461e-a658-4880b3ea174d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:57.960820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f163978-01d1-461e-a658-4880b3ea174d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:57.962298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1342/2645] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:20:57.968728 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3223e-1ab3-4223-bf89-0f7aaf7ddf6b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:57.969228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3223e-1ab3-4223-bf89-0f7aaf7ddf6b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:57.969541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3223e-1ab3-4223-bf89-0f7aaf7ddf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:57.991058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8bd3223e-1ab3-4223-bf89-0f7aaf7ddf6b None None] MFA Rules not processed for user `5fda0e3ee3624ac19de85bb3db30e2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:57.992382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8bd3223e-1ab3-4223-bf89-0f7aaf7ddf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:58.033191 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:20:58.056400 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1304/2646] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:20:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:58.068924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f163978-01d1-461e-a658-4880b3ea174d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:58.071444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f163978-01d1-461e-a658-4880b3ea174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:58.085723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f163978-01d1-461e-a658-4880b3ea174d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:58.089785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddb97f-3afa-499f-9f70-083af2bb12ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:58.090511 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb97f-3afa-499f-9f70-083af2bb12ac service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:58.090804 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb97f-3afa-499f-9f70-083af2bb12ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:58.091177 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb97f-3afa-499f-9f70-083af2bb12ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:58.098940 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddb97f-3afa-499f-9f70-083af2bb12ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:58.100373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddb97f-3afa-499f-9f70-083af2bb12ac service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:58.108484 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1343/2647] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:20:58.208732 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d602-6934-41b0-932a-29394ef81bfd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:58.209042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d602-6934-41b0-932a-29394ef81bfd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:58.209306 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d602-6934-41b0-932a-29394ef81bfd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:58.210671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1305/2648] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jul 28 01:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 01:20:58.218730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:58.219140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:58.219474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:58.241581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] MFA Rules not processed for user `8f2f77d4f4c44890ac1ab88ff66100bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:20:58.242824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:20:58.253816 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1344/2649] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jul 28 01:20:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:20:58.266127 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131d602-6934-41b0-932a-29394ef81bfd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:20:58.278761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131d602-6934-41b0-932a-29394ef81bfd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:20:58.282992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:20:58.283629 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:20:58.283913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:20:58.284192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:20:58.295544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:20:58.297175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:20:58.309166 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1306/2650] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:20:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:04.758607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:04.829567 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda02330-39ec-4de8-85dd-c081a7c477bf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:04.834775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:04.835462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:04.835749 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:04.836033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] PATH_INFO: `/v3/users/3bfaad5e15074146b5bf7e51db52ccba` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:04.837280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bfaad5e15074146b5bf7e51db52ccba)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:04.839271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:04.891423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:04.893184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bfaad5e15074146b5bf7e51db52ccba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:04.894118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bfaad5e15074146b5bf7e51db52ccba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:04.895086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bfaad5e15074146b5bf7e51db52ccba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:04.902475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bfaad5e15074146b5bf7e51db52ccba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:04.914786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bfaad5e15074146b5bf7e51db52ccba'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:04.930762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1345/2651] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:04 2022] DELETE /identity/v3/users/3bfaad5e15074146b5bf7e51db52ccba => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:05.141341 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:05.159610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30603604-4bf9-4cdc-aaeb-f8efc9d301e2 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:05.233352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:05.234170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:05.234453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:05.234865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] PATH_INFO: `/v3/projects/f9d1dce0e819423b97d775e29216df21` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:05.236347 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d1dce0e819423b97d775e29216df21)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:05.238415 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:05.287662 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d1dce0e819423b97d775e29216df21', 'request_id': 'req-bf45d596-1676-41cd-9397-7a1e3fc60561'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.295624 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d1dce0e819423b97d775e29216df21', 'request_id': 'req-bf45d596-1676-41cd-9397-7a1e3fc60561'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.301528 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d1dce0e819423b97d775e29216df21', 'request_id': 'req-bf45d596-1676-41cd-9397-7a1e3fc60561'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.312532 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] The token cache is being invalidate because project f9d1dce0e819423b97d775e29216df21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:05.312821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.313445 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bf45d596-1676-41cd-9397-7a1e3fc60561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.315055 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1307/2652] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:05 2022] DELETE /identity/v3/projects/f9d1dce0e819423b97d775e29216df21 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:05.319433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:05.320863 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:05.334670 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded3063-8137-4885-b892-05ff7aa7f88f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:05.379031 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:05.379437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:05.379743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:05.400776 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] MFA Rules not processed for user `82a7db7a0f544b3eb9487bf00adbb742`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:05.401937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:05.406339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:05.407182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:05.407597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:05.407900 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] PATH_INFO: `/v3/users/1b26073d75034f22aacc9935ceca2fc1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:05.409183 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b26073d75034f22aacc9935ceca2fc1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:05.411244 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:05.464807 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1308/2653] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:05.471205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b26073d75034f22aacc9935ceca2fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.475993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b26073d75034f22aacc9935ceca2fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.476865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b26073d75034f22aacc9935ceca2fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.485308 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b26073d75034f22aacc9935ceca2fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.497582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f9e7ce26-6d95-4378-86bf-9b4d63aa2e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b26073d75034f22aacc9935ceca2fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.512166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1346/2654] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:05 2022] DELETE /identity/v3/users/1b26073d75034f22aacc9935ceca2fc1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:05.694797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:05.696494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:05.716465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ec693-85e6-4cf0-ab77-acb3b6b28234 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:05.720465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:05.780566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:05.781234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:05.781551 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:05.781834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] PATH_INFO: `/v3/projects/dad16612be6441a88785c89c692f3ed1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:05.783085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad16612be6441a88785c89c692f3ed1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:05.785037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:05.831254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad16612be6441a88785c89c692f3ed1', 'request_id': 'req-73058b43-b893-4222-be1f-19737be7750f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.839009 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad16612be6441a88785c89c692f3ed1', 'request_id': 'req-73058b43-b893-4222-be1f-19737be7750f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.845211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad16612be6441a88785c89c692f3ed1', 'request_id': 'req-73058b43-b893-4222-be1f-19737be7750f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.856922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] The token cache is being invalidate because project dad16612be6441a88785c89c692f3ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:05.857292 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.858002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73058b43-b893-4222-be1f-19737be7750f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:05.859637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1309/2655] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:05 2022] DELETE /identity/v3/projects/dad16612be6441a88785c89c692f3ed1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:05.864708 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:05.865233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:05.865336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:05.885268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:05.886489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:05.925705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:05.950723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1347/2656] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:05.959317 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:05.959692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:05.960252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:05.983469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:05.984816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:05.993001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1310/2657] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:05.997421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.012323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30087d2-6d9a-4665-9668-c4f502e7c541 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.018624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.019436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.019717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.019990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.020737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:21:06.021062 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.022775 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.036566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1348/2658] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:21:06.041153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.051463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3aba7-603f-4ee4-9089-c48c45c8c5a3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.056470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.057120 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.057409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.057693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.058421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.060502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.094906 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1311/2659] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:06 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:21:06.098415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.113805 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0a95b-4909-457b-bce4-f647796014b2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.174914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.175558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.175840 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.176116 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.176898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.178854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.202930 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1349/2660] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:06 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:06.207037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.218637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef41dc3-82a3-49af-a956-3bfbe194c06d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.226327 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.227038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.227385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.227733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.228581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.230102 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.244641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1312/2661] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:06.245898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.258458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecc4db-e0ac-42b0-a956-65e3738710f1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.263245 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a28e84-0910-4a17-87f7-b01acef3f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.263870 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a28e84-0910-4a17-87f7-b01acef3f8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.264155 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a28e84-0910-4a17-87f7-b01acef3f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.264427 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a28e84-0910-4a17-87f7-b01acef3f8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.265280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a28e84-0910-4a17-87f7-b01acef3f8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.266895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a28e84-0910-4a17-87f7-b01acef3f8b7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.281424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1350/2662] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:21:06.286004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.298667 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159b24-11c3-43b2-9675-50ed2db490cc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.304353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.305123 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.305410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.305723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] PATH_INFO: `/v3/projects/bf2113e05e71459bafb08db9a995e336/users/c1cc0cf575b14474952307b64c34bed3/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.322693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2113e05e71459bafb08db9a995e336, user_id=c1cc0cf575b14474952307b64c34bed3, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.325395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.342369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1313/2663] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:06 2022] PUT /identity/v3/projects/bf2113e05e71459bafb08db9a995e336/users/c1cc0cf575b14474952307b64c34bed3/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:06.353982 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.354302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.354674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.386921 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] MFA Rules not processed for user `c1cc0cf575b14474952307b64c34bed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:06.388262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:06.462938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1351/2664] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:06.467529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.519157 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:06.550935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196ac08-c602-48df-b971-6bc6c63cee76 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.556228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.556899 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.557185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.557463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.558155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.560106 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.588602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1314/2665] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:06 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:06.592795 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.604280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303fb2-2e60-4a98-bc42-8db847402413 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.640418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:06.670233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.670819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.671156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.671440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.672128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.674251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.696551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1352/2666] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:06 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:21:06.700642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.711485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fcee4-9640-4f90-a84c-65301295ed69 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.717422 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.718187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.718472 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.718748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.719519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.721566 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.738800 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1315/2667] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:06.740391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.755914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8cf19-5242-4eb5-b762-3b575dddc873 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.761888 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.762597 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.762922 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.763205 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] PATH_INFO: `/v3/projects/b53c8d44e54c45afa4a4ded99c4a9d84/users/3252e6000fe4407aa5f87d0f6b541be6/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.781792 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53c8d44e54c45afa4a4ded99c4a9d84, user_id=3252e6000fe4407aa5f87d0f6b541be6, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.784526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.799619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1353/2668] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:06 2022] PUT /identity/v3/projects/b53c8d44e54c45afa4a4ded99c4a9d84/users/3252e6000fe4407aa5f87d0f6b541be6/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:06.803778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.814079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1f3f-6282-417e-90e7-3f306714da08 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.872494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606ef92-59af-4470-96d3-31a13998ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.873138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606ef92-59af-4470-96d3-31a13998ac6a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.873424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606ef92-59af-4470-96d3-31a13998ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.873700 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606ef92-59af-4470-96d3-31a13998ac6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.874411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606ef92-59af-4470-96d3-31a13998ac6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.876069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606ef92-59af-4470-96d3-31a13998ac6a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.889593 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1316/2669] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:06.893964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:06.905956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce145f-aadb-409c-91f9-5935677deac0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:06.911159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:06.911850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.912138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.912417 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] PATH_INFO: `/v3/projects/b53c8d44e54c45afa4a4ded99c4a9d84/users/3252e6000fe4407aa5f87d0f6b541be6/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.914598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53c8d44e54c45afa4a4ded99c4a9d84, user_id=3252e6000fe4407aa5f87d0f6b541be6, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:06.917371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:06.931843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1354/2670] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:06 2022] PUT /identity/v3/projects/b53c8d44e54c45afa4a4ded99c4a9d84/users/3252e6000fe4407aa5f87d0f6b541be6/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:06.944025 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:06.944275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:06.944459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:06.992895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] MFA Rules not processed for user `3252e6000fe4407aa5f87d0f6b541be6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:06.994643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:07.097904 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1317/2671] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:07.110862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:07.112722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:07.193963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008ad4-639e-4234-a3ad-6487f9666a71 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:07.201914 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:07.202599 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:07.202919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:07.203204 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:07.270242 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:07.271953 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:07.282168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1355/2672] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:11.724067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:11.725524 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:11.737760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15896d5-bbb8-41a5-b72a-1d6f7ee05445 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:11.742165 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d3041-15ce-44c2-a647-587e9a84d5e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:11.742599 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d3041-15ce-44c2-a647-587e9a84d5e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:11.742913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d3041-15ce-44c2-a647-587e9a84d5e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:11.743197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d3041-15ce-44c2-a647-587e9a84d5e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:11.754289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d3041-15ce-44c2-a647-587e9a84d5e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:11.756261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d3041-15ce-44c2-a647-587e9a84d5e9 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:11.766996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1318/2673] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:16.772978 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:16.819705 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:16.837604 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bff23-1216-4f92-91d1-8d7c8392788d service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:16.843899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:16.844600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:16.844883 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:16.845161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] PATH_INFO: `/v3/users/c1cc0cf575b14474952307b64c34bed3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:16.846404 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1cc0cf575b14474952307b64c34bed3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:16.848357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:16.899168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1cc0cf575b14474952307b64c34bed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:16.899842 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1cc0cf575b14474952307b64c34bed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:16.900450 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1cc0cf575b14474952307b64c34bed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:16.907618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1cc0cf575b14474952307b64c34bed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:16.919651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1cc0cf575b14474952307b64c34bed3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:16.934651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1356/2674] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:16 2022] DELETE /identity/v3/users/c1cc0cf575b14474952307b64c34bed3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:16.939370 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:16.939913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:16.939913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:16.959046 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:16.960262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:16.969852 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:17.023559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1319/2675] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:17.033598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:17.035154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:17.096045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fe637-fa75-45f6-af71-d48cf51e1a73 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:17.100815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:17.101452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:17.101739 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:17.102015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:17.112615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:17.114061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:17.124314 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1357/2676] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:17.232585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:17.234053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:17.249611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe071-39e5-4ba0-9d52-750044b3b3b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:17.254470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:17.255530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:17.255817 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:17.256093 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:17.258155 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:17.272982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:17.274908 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:17.286384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1320/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:17.437537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:17.440475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:17.455920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3ecaa-a1de-4423-a585-a2b15fd52799 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:17.464983 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:17.465698 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:17.465979 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:17.466328 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] PATH_INFO: `/v3/projects/bf2113e05e71459bafb08db9a995e336` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:17.467632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2113e05e71459bafb08db9a995e336)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:17.469846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:17.521563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2113e05e71459bafb08db9a995e336', 'request_id': 'req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.528471 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2113e05e71459bafb08db9a995e336', 'request_id': 'req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.536767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2113e05e71459bafb08db9a995e336', 'request_id': 'req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.538117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:17.549867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] The token cache is being invalidate because project bf2113e05e71459bafb08db9a995e336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:17.550462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.551148 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.552694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1358/2678] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:17 2022] DELETE /identity/v3/projects/bf2113e05e71459bafb08db9a995e336 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:17.556897 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:17.567151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150e092-cabf-4020-9b3a-114707e192f1 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:17.633385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:17.634216 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:17.634634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:17.635060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] PATH_INFO: `/v3/users/3252e6000fe4407aa5f87d0f6b541be6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:17.636409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3252e6000fe4407aa5f87d0f6b541be6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:17.638476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:17.691682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3252e6000fe4407aa5f87d0f6b541be6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.699788 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3252e6000fe4407aa5f87d0f6b541be6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.700418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3252e6000fe4407aa5f87d0f6b541be6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.714041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3252e6000fe4407aa5f87d0f6b541be6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.721419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4eab9700-d9c1-44c5-90b5-f8c01035d153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3252e6000fe4407aa5f87d0f6b541be6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:17.727738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1321/2679] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:17 2022] DELETE /identity/v3/users/3252e6000fe4407aa5f87d0f6b541be6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:21:18.168241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.170478 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:18.184916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25ea08-2370-48fa-a1b8-c6b2316cbe2d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.253296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.253974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.254262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.254550 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] PATH_INFO: `/v3/projects/b53c8d44e54c45afa4a4ded99c4a9d84` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.256341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53c8d44e54c45afa4a4ded99c4a9d84)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.258285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.340186 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53c8d44e54c45afa4a4ded99c4a9d84', 'request_id': 'req-0cf54f68-225c-4bf1-924b-df3af32d7b37'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:18.350696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53c8d44e54c45afa4a4ded99c4a9d84', 'request_id': 'req-0cf54f68-225c-4bf1-924b-df3af32d7b37'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:18.365581 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53c8d44e54c45afa4a4ded99c4a9d84', 'request_id': 'req-0cf54f68-225c-4bf1-924b-df3af32d7b37'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:18.384741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] The token cache is being invalidate because project b53c8d44e54c45afa4a4ded99c4a9d84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:18.385033 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:18.385778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0cf54f68-225c-4bf1-924b-df3af32d7b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:18.387357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1359/2680] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:18 2022] DELETE /identity/v3/projects/b53c8d44e54c45afa4a4ded99c4a9d84 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:18.393565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.394037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.394132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.416332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:18.417942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:18.450064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:18.552113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1322/2681] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:18.625060 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788766d-b51e-495a-9217-502909fa62ad None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.626272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788766d-b51e-495a-9217-502909fa62ad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.626672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788766d-b51e-495a-9217-502909fa62ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.649778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b788766d-b51e-495a-9217-502909fa62ad None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:18.651231 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b788766d-b51e-495a-9217-502909fa62ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:18.664274 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.672551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1360/2682] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:18.674878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630771b2-372c-4e1b-862a-b9b59ad3fddc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.679791 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.680541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.680925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.681277 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.682112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:21:18.682414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.684147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.698873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1323/2683] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:21:18.701392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788766d-b51e-495a-9217-502909fa62ad None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.710820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788766d-b51e-495a-9217-502909fa62ad None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.715497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.716120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.716403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.716677 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.717362 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.719348 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.749202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1361/2684] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:18 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:21:18.754076 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.765476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da79373-619c-4b83-909f-6f8a5f8a06d2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.824862 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.825523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.825799 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.826068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.826725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.828745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.854266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1324/2685] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:18 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:18.858364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.869552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f420e1-4b9c-4b81-abb1-3ffd368c60ac admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.876285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.876959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.877276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.877574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.878298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.880439 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.891375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1362/2686] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:18.895845 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.906387 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e12ec6-8d43-4dc0-806b-7154f137f3f6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.912112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64b473-df1b-42b7-b227-8d9f4a21cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.912378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64b473-df1b-42b7-b227-8d9f4a21cbcb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.912714 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64b473-df1b-42b7-b227-8d9f4a21cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.912992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64b473-df1b-42b7-b227-8d9f4a21cbcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.913748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64b473-df1b-42b7-b227-8d9f4a21cbcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.915488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64b473-df1b-42b7-b227-8d9f4a21cbcb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.927586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1325/2687] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:21:18.931898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:18.942296 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7747c7-4a6b-4ab1-9820-c3cba1e4ed98 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:18.947451 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:18.948720 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:18.949072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:18.949503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] PATH_INFO: `/v3/projects/c8c3719369a64c2ea5d5231c873d839e/users/fd377e67e8624aaab526cabcf306e466/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:18.970499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c3719369a64c2ea5d5231c873d839e, user_id=fd377e67e8624aaab526cabcf306e466, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:18.973986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:18.992341 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1363/2688] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:18 2022] PUT /identity/v3/projects/c8c3719369a64c2ea5d5231c873d839e/users/fd377e67e8624aaab526cabcf306e466/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:21:19.003315 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.003785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.004196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.036335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] MFA Rules not processed for user `fd377e67e8624aaab526cabcf306e466`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:19.037678 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:19.109966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1326/2689] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:19.115586 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.136516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:19.192366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c735d-89b2-4796-b0b1-062fe26edf2d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.198368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.198995 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.199285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.199574 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.200312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.202209 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.238391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1364/2690] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:19 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:19.242383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.255175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58565-0bde-40d3-8ae0-c6edc591fe01 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.270355 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:19.331206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.331857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.332142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.332421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.333096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.335202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.362802 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1327/2691] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:19 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:19.366855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.377935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4635a76-6928-4186-82d4-839377399792 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.383277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.383925 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.384278 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.384623 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.385397 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.387086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.398610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1365/2692] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:21:19.402663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.413224 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70c51e-4e19-478e-a7bc-ed33e5c3a5bc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.418338 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22c0ff-8d24-4efe-86e6-35233391ea9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.419012 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c0ff-8d24-4efe-86e6-35233391ea9e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.419272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c0ff-8d24-4efe-86e6-35233391ea9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.419582 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c0ff-8d24-4efe-86e6-35233391ea9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.420293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22c0ff-8d24-4efe-86e6-35233391ea9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.421916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22c0ff-8d24-4efe-86e6-35233391ea9e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.436465 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1328/2693] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:19.440903 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.455117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f9168-3ca7-41ee-b103-25fead3e20f7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.460460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.461364 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.461846 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.462213 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] PATH_INFO: `/v3/projects/3953dabdd95a48ef8ea968d99f112561/users/ce0fdf57965440df8e1c52ce16863ce3/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.484582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3953dabdd95a48ef8ea968d99f112561, user_id=ce0fdf57965440df8e1c52ce16863ce3, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.487400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.501493 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1366/2694] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:19 2022] PUT /identity/v3/projects/3953dabdd95a48ef8ea968d99f112561/users/ce0fdf57965440df8e1c52ce16863ce3/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:19.512410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.512704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.512985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.547513 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] MFA Rules not processed for user `ce0fdf57965440df8e1c52ce16863ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:19.548793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:19.619521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1329/2695] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:19.623916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.699322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422672ca-e450-4cbc-a14d-f8a489452bb2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.704766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.705402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.705687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.705965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.706667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.708621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.742898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1367/2696] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:19 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:21:19.747126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.759103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b686a-b5ef-41b5-9edf-a952772e0ea6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.825943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.826683 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.827067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.827409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.828162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.830374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.857684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1330/2697] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:19 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:19.858934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.870488 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299f7e8-11c0-4efa-b97e-aa2a1e1361d8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.877938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.878504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.878872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.879153 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.881890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.881890 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.894646 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1368/2698] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:19.899986 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.915872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddafcf-47da-4f04-91d5-c72953dac809 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:19.921396 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:19.922090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:19.922372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:19.922645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] PATH_INFO: `/v3/projects/2381c804ddac4f5d9cc46b855055d699/users/014460dd8267406e86a9a64e77d454ea/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:19.935853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:19.940126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2381c804ddac4f5d9cc46b855055d699, user_id=014460dd8267406e86a9a64e77d454ea, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:19.943026 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:19.958427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1331/2699] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:19 2022] PUT /identity/v3/projects/2381c804ddac4f5d9cc46b855055d699/users/014460dd8267406e86a9a64e77d454ea/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:19.962251 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:19.974234 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4696-fa1a-4d21-9644-fa539b6050d6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:20.027844 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:20.050614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01f90c-bfad-49c2-9a65-62d2b5277a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:20.051346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01f90c-bfad-49c2-9a65-62d2b5277a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:20.051624 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01f90c-bfad-49c2-9a65-62d2b5277a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:20.051901 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01f90c-bfad-49c2-9a65-62d2b5277a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:20.052615 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01f90c-bfad-49c2-9a65-62d2b5277a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:20.054214 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01f90c-bfad-49c2-9a65-62d2b5277a65 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:20.068460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1369/2700] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:20.070787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:20.082806 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8ad49-41c6-4e6b-9484-9c2a002ef476 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:20.087636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:20.088331 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:20.088653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:20.088895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] PATH_INFO: `/v3/projects/2381c804ddac4f5d9cc46b855055d699/users/014460dd8267406e86a9a64e77d454ea/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:20.091103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2381c804ddac4f5d9cc46b855055d699, user_id=014460dd8267406e86a9a64e77d454ea, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:20.094176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:20.107333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1332/2701] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:20 2022] PUT /identity/v3/projects/2381c804ddac4f5d9cc46b855055d699/users/014460dd8267406e86a9a64e77d454ea/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:20.119215 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:20.119866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:20.120170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:20.158682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] MFA Rules not processed for user `014460dd8267406e86a9a64e77d454ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:20.160049 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:20.233184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1370/2702] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:20.245403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:20.247321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:20.319996 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d3e9-42ee-41f0-9403-eb767735787a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:20.325066 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:20.325949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:20.326254 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:20.326663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:20.402406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:20.404463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:20.418078 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1333/2703] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:20.707537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:20.709382 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:20.789652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121284c5-1ee0-4c47-bfe2-0eb2456cbbcf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:20.794255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:20.794949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:20.795241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:20.795526 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] PATH_INFO: `/v3/users/166cef282cb24a5aaef0ca8b28f02e56` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:20.796706 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166cef282cb24a5aaef0ca8b28f02e56)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:20.798680 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:20.855134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166cef282cb24a5aaef0ca8b28f02e56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:20.855134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166cef282cb24a5aaef0ca8b28f02e56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:20.855629 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166cef282cb24a5aaef0ca8b28f02e56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:20.872175 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166cef282cb24a5aaef0ca8b28f02e56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:20.878195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:20.895375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7f10a6d3-4b6b-4925-b973-841c394cec11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166cef282cb24a5aaef0ca8b28f02e56'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:20.910378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1371/2704] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:20 2022] DELETE /identity/v3/users/166cef282cb24a5aaef0ca8b28f02e56 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:21.099689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.111470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a0b30-592c-4fa9-a84f-0641d401a448 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.172363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.172798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.173080 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.173375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] PATH_INFO: `/v3/projects/6e3df8020ac0427bae99a7fc30bdfa48` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.174560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3df8020ac0427bae99a7fc30bdfa48)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.176487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.212689 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:21.223499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3df8020ac0427bae99a7fc30bdfa48', 'request_id': 'req-e650915c-9ecc-473f-a4bf-58ebbf115729'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:21.231505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3df8020ac0427bae99a7fc30bdfa48', 'request_id': 'req-e650915c-9ecc-473f-a4bf-58ebbf115729'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:21.237003 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3df8020ac0427bae99a7fc30bdfa48', 'request_id': 'req-e650915c-9ecc-473f-a4bf-58ebbf115729'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:21.248042 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] The token cache is being invalidate because project 6e3df8020ac0427bae99a7fc30bdfa48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:21.248325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:21.248992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e650915c-9ecc-473f-a4bf-58ebbf115729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:21.250598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1334/2705] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:21 2022] DELETE /identity/v3/projects/6e3df8020ac0427bae99a7fc30bdfa48 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:21.256206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144dd61-0715-4523-a826-0a0086a801df None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.256726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144dd61-0715-4523-a826-0a0086a801df None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.256831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144dd61-0715-4523-a826-0a0086a801df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.277474 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f144dd61-0715-4523-a826-0a0086a801df None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:21.278701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f144dd61-0715-4523-a826-0a0086a801df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:21.343380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1372/2706] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:21:21.351973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.352261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.352541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.372761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:21.373972 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:21.386511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1335/2707] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:21.391503 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144dd61-0715-4523-a826-0a0086a801df None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.409323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144dd61-0715-4523-a826-0a0086a801df None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.415070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.415742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.416194 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.416521 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.417323 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:21:21.417637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.419291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.432371 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1373/2708] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:21:21.438610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.448738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b198b8-5d8e-45bb-ace4-f628ec645860 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.455101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.455743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.456027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.456301 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.456976 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.458905 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.503964 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1336/2709] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:21 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:21.508742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.519465 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97340386-f53f-40a9-b7da-8a1fe31ecda7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.580402 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.581038 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.581346 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.581614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.582256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.584256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.606258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1374/2710] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:21 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:21.610495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.620763 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab59bf-4f23-4a49-8abc-a11677ede395 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.626133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.626756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.627068 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.627351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.628054 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.629657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.640596 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1337/2711] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:21.646201 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.659711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0ecfb-20bb-43f3-96fe-fbfb9083b7bb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.664354 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257b38b-f4d7-4409-a104-20ceca76d38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.664976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257b38b-f4d7-4409-a104-20ceca76d38d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.665262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257b38b-f4d7-4409-a104-20ceca76d38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.665536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257b38b-f4d7-4409-a104-20ceca76d38d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.666238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257b38b-f4d7-4409-a104-20ceca76d38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.667894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257b38b-f4d7-4409-a104-20ceca76d38d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.680161 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1375/2712] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:21.684787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.695284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b35a1-920a-4823-8b2b-55210a32e982 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.700060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.700733 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.701021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.701313 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] PATH_INFO: `/v3/projects/adb91176f5f74f1199b2fc27c248cecd/users/6b069c6a7f1e4241816e5c91874cbcd6/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.716595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb91176f5f74f1199b2fc27c248cecd, user_id=6b069c6a7f1e4241816e5c91874cbcd6, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.719386 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.734500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1338/2713] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:21 2022] PUT /identity/v3/projects/adb91176f5f74f1199b2fc27c248cecd/users/6b069c6a7f1e4241816e5c91874cbcd6/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:21.739977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.740270 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.740570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.761031 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:21.762361 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:21.811524 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:21.832203 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1376/2714] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:21.842504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:21.844682 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:21.906354 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6ded2-1530-487e-9bb2-96af62e035c8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:21.910460 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:21.911657 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:21.911946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:21.912231 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:21.925412 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:21.926872 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:21.936769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1339/2715] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:22.090655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:22.092208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:22.156376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae19b94-6c4b-4a33-a488-7b0ae36a2c23 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:22.160396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:22.161040 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:22.161324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:22.161606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:22.163504 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:22.227233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:22.228694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a68a1e-029c-4ed2-8ea5-35cbf68721f3 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:22.238423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1377/2716] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:22.858454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:22.877840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fc2ab-23fb-40b4-b8df-acfaab48defc service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:22.882502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:22.883206 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:22.883774 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:22.884207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:22.886717 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:22.958185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:22.961547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:22.963440 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:22.978235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1340/2717] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:26.565441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ade48-d83b-469c-b9cf-84db9b168d1a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:26.565991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ade48-d83b-469c-b9cf-84db9b168d1a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:26.566106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ade48-d83b-469c-b9cf-84db9b168d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:26.598285 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ac6ade48-d83b-469c-b9cf-84db9b168d1a None None] MFA Rules not processed for user `6b069c6a7f1e4241816e5c91874cbcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:26.599556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ac6ade48-d83b-469c-b9cf-84db9b168d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:26.691759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1378/2718] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:26.706351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:26.713114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:26.813000 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bca0b-a60d-4ff4-ae26-826e772ababe service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:26.818951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:26.819654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:26.819940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:26.820227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:26.829530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:26.831278 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:26.845966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1341/2719] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:28.205598 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:28.206130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:28.206130 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:28.225866 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] MFA Rules not processed for user `6b069c6a7f1e4241816e5c91874cbcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:28.227206 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:28.235317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1379/2720] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:28.405368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:28.415959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a38b19-5dba-48a9-bde7-959018cca1a8 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:28.420606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:28.421243 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:28.421530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:28.421968 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:28.424091 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:28.502536 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:28.504282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e77893-7f35-43d6-8639-83c92588ffa4 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:28.535433 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1342/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:29.171010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:29.172576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:29.188085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb50d0-2b97-4e95-8a31-85d09d255f70 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:29.193067 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:29.193726 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:29.194053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:29.194499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] PATH_INFO: `/v3/users/fd377e67e8624aaab526cabcf306e466` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:29.195857 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd377e67e8624aaab526cabcf306e466)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:29.197892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:29.220783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:29.251080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd377e67e8624aaab526cabcf306e466'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:29.253557 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd377e67e8624aaab526cabcf306e466'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:29.254220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd377e67e8624aaab526cabcf306e466'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:29.262026 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd377e67e8624aaab526cabcf306e466'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:29.277647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd377e67e8624aaab526cabcf306e466'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:29.296126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1380/2722] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:29 2022] DELETE /identity/v3/users/fd377e67e8624aaab526cabcf306e466 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:29.299017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:29.300219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:29.300745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:29.323946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:29.325183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:29.411376 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1343/2723] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:29.422906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:29.425124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:29.524365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f4abb-a5da-4914-9055-57c036357df5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:29.533778 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:29.534446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:29.534736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:29.540632 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:29.561093 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:29.563389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:29.576388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1381/2724] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:29.973208 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:29.974917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:30.003456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119421c2-33db-4ba0-8265-bfb1544ae0c0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:30.018672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:30.019761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:30.020001 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:30.025117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] PATH_INFO: `/v3/projects/c8c3719369a64c2ea5d5231c873d839e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:30.028005 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c3719369a64c2ea5d5231c873d839e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:30.030625 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:30.054764 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:30.100705 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c3719369a64c2ea5d5231c873d839e', 'request_id': 'req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.111484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c3719369a64c2ea5d5231c873d839e', 'request_id': 'req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.119469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c3719369a64c2ea5d5231c873d839e', 'request_id': 'req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.138437 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] The token cache is being invalidate because project c8c3719369a64c2ea5d5231c873d839e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:30.138884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.139634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.144636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1344/2725] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:29 2022] DELETE /identity/v3/projects/c8c3719369a64c2ea5d5231c873d839e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:30.145986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:30.156159 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e574d6-114c-48e1-b521-41e464896812 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:30.219288 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:30.220272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:30.220732 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:30.221176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] PATH_INFO: `/v3/users/ce0fdf57965440df8e1c52ce16863ce3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:30.223073 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0fdf57965440df8e1c52ce16863ce3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:30.225283 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:30.272630 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:30.278618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0fdf57965440df8e1c52ce16863ce3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.279405 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0fdf57965440df8e1c52ce16863ce3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.280490 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0fdf57965440df8e1c52ce16863ce3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.288733 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0fdf57965440df8e1c52ce16863ce3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.305415 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0fdf57965440df8e1c52ce16863ce3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.320849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1382/2726] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:30 2022] DELETE /identity/v3/users/ce0fdf57965440df8e1c52ce16863ce3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:30.633207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:30.636517 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:30.655987 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e51ae8-90e0-45e4-ab4b-273c47eb0ce5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:30.746793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:30.748287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:30.748747 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:30.749172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] PATH_INFO: `/v3/projects/3953dabdd95a48ef8ea968d99f112561` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:30.750598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3953dabdd95a48ef8ea968d99f112561)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:30.752866 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:30.818915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3953dabdd95a48ef8ea968d99f112561', 'request_id': 'req-f04c5c58-621a-4ba3-9639-97b9941b3509'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.827409 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3953dabdd95a48ef8ea968d99f112561', 'request_id': 'req-f04c5c58-621a-4ba3-9639-97b9941b3509'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.833718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3953dabdd95a48ef8ea968d99f112561', 'request_id': 'req-f04c5c58-621a-4ba3-9639-97b9941b3509'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.845420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] The token cache is being invalidate because project 3953dabdd95a48ef8ea968d99f112561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:30.845720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.846410 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.847966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1345/2727] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:30 2022] DELETE /identity/v3/projects/3953dabdd95a48ef8ea968d99f112561 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:21:30.852061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:30.853958 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:30.869156 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec7224-4aff-4137-89d6-39b234183f5f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:30.939963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:30.940687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:30.940971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:30.941246 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] PATH_INFO: `/v3/users/014460dd8267406e86a9a64e77d454ea` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:30.942461 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=014460dd8267406e86a9a64e77d454ea)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:30.944495 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:30.997228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014460dd8267406e86a9a64e77d454ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.998081 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014460dd8267406e86a9a64e77d454ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:30.998764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014460dd8267406e86a9a64e77d454ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.006754 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014460dd8267406e86a9a64e77d454ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.019909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-803502bd-98df-4255-86a2-d930454549ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014460dd8267406e86a9a64e77d454ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.039102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1383/2728] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:30 2022] DELETE /identity/v3/users/014460dd8267406e86a9a64e77d454ea => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:21:31.324411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.326024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:31.339196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c5c58-621a-4ba3-9639-97b9941b3509 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.370720 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:31.408107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.408768 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.409051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.409330 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] PATH_INFO: `/v3/projects/2381c804ddac4f5d9cc46b855055d699` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.410537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2381c804ddac4f5d9cc46b855055d699)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.412606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.464805 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2381c804ddac4f5d9cc46b855055d699', 'request_id': 'req-61f46c81-da73-4d81-a29f-8a1d86faaf18'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.473677 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2381c804ddac4f5d9cc46b855055d699', 'request_id': 'req-61f46c81-da73-4d81-a29f-8a1d86faaf18'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.479766 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2381c804ddac4f5d9cc46b855055d699', 'request_id': 'req-61f46c81-da73-4d81-a29f-8a1d86faaf18'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.491052 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] The token cache is being invalidate because project 2381c804ddac4f5d9cc46b855055d699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:31.491344 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.492177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-61f46c81-da73-4d81-a29f-8a1d86faaf18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:31.494047 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1346/2729] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:31 2022] DELETE /identity/v3/projects/2381c804ddac4f5d9cc46b855055d699 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:31.500400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.500894 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.501020 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.528047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:31.529695 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:31.572425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:31.605385 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1384/2730] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:31.614568 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.614891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.615176 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.638457 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:31.639830 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:31.652637 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1347/2731] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:31.657238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.667935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de180d-fd44-480a-b782-b75c0acf77ac None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.673691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.674374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.674662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.675016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.675723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:21:31.676024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.677649 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.691442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1385/2732] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:21:31.700652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.710590 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c90f1d-6d57-4948-b166-193f7de4aa1e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.717699 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.718735 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.719008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.719479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.720173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.722181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.755187 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1348/2733] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:21:31 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:31.756712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.768134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188560a-0d2d-4568-a3ae-30bc94dde6ec admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.827996 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.828742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.829066 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.829384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.830087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.832085 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.855522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1386/2734] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:21:31 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:21:31.859538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.870829 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081269b4-af42-4671-9d26-c7a5697de3b1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.877234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.877884 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.878170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.878467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.879228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.881236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.892959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1349/2735] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:31.898012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.910499 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fff57-adb5-4466-ad2a-0836a9cbbddd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.919566 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087230b0-88d6-4496-85c8-99e097876f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.920322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087230b0-88d6-4496-85c8-99e097876f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.920424 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087230b0-88d6-4496-85c8-99e097876f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.920653 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087230b0-88d6-4496-85c8-99e097876f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.921375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087230b0-88d6-4496-85c8-99e097876f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.923776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087230b0-88d6-4496-85c8-99e097876f9f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.940233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1387/2736] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:21:31.941963 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:31.953002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2ac8-ce78-45c6-8a14-401c946b9eeb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:31.957864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:31.958569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:31.958879 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:31.959153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] PATH_INFO: `/v3/projects/7f9fcd161fc74a9891adc66a1d5d8bca/users/966913b7e0af4fb687db63ebdaae281a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:31.976358 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9fcd161fc74a9891adc66a1d5d8bca, user_id=966913b7e0af4fb687db63ebdaae281a, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:31.979058 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:31.994523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1350/2737] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:21:31 2022] PUT /identity/v3/projects/7f9fcd161fc74a9891adc66a1d5d8bca/users/966913b7e0af4fb687db63ebdaae281a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:32.006696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:32.007024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:32.007297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:32.036235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] MFA Rules not processed for user `966913b7e0af4fb687db63ebdaae281a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:32.037489 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:32.111406 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1388/2738] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:32.123877 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:32.126654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:32.188160 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f0213-ed24-405e-8b39-8a94e7b71e61 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:32.192227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada07d8-3e62-4214-b0c2-ac6500443e3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:32.192860 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada07d8-3e62-4214-b0c2-ac6500443e3b service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:32.193141 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada07d8-3e62-4214-b0c2-ac6500443e3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:32.193420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada07d8-3e62-4214-b0c2-ac6500443e3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:32.202613 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada07d8-3e62-4214-b0c2-ac6500443e3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:32.205075 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada07d8-3e62-4214-b0c2-ac6500443e3b service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:32.213881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1351/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:39.887848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:39.889517 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:39.932848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:39.991034 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ce15e-343a-4c9c-b888-e30ff91e7ea5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:39.996238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:39.996885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:39.997176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:39.997458 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:39.999373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:40.011704 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:40.013453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:40.023783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1389/2740] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:21:40.849801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af885-64e4-4cbb-ab6b-74db423a9c37 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:40.850366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af885-64e4-4cbb-ab6b-74db423a9c37 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:40.850366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af885-64e4-4cbb-ab6b-74db423a9c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:40.857635 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:40.871212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d74af885-64e4-4cbb-ab6b-74db423a9c37 None None] MFA Rules not processed for user `d9cb42db2f9a4b80b17bef0a86945a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:40.873059 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d74af885-64e4-4cbb-ab6b-74db423a9c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:40.938540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1352/2741] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:43.564764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:43.668400 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956b085-3d43-4f4e-92f8-fefe5bad6f57 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:43.674618 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:43.675611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:43.676056 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:43.676560 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] PATH_INFO: `/v3/users/966913b7e0af4fb687db63ebdaae281a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:43.678019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=966913b7e0af4fb687db63ebdaae281a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:43.680154 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:43.778467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966913b7e0af4fb687db63ebdaae281a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:43.785926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966913b7e0af4fb687db63ebdaae281a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:43.787339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966913b7e0af4fb687db63ebdaae281a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:43.796558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966913b7e0af4fb687db63ebdaae281a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:43.810702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966913b7e0af4fb687db63ebdaae281a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:43.834664 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1390/2742] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:21:43 2022] DELETE /identity/v3/users/966913b7e0af4fb687db63ebdaae281a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:43.839418 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:43.839948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:43.840201 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:43.870601 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:21:43.872284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:43.951112 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1353/2743] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:21:43.961668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:43.963375 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:44.006089 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:44.051297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae6b57-bebb-46fc-b42c-8965c004d1bb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:44.055322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:44.056016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:44.056302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:44.056583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:44.067927 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:44.069441 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:44.080086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1391/2744] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:21:44.644923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:21:44.646580 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:21:44.659786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788210f-c449-4638-b536-4f8ad3ba90cb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:21:44.664719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:21:44.665388 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:21:44.665691 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:21:44.665993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] PATH_INFO: `/v3/projects/7f9fcd161fc74a9891adc66a1d5d8bca` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:21:44.667235 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9fcd161fc74a9891adc66a1d5d8bca)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:21:44.669359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:21:44.721361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9fcd161fc74a9891adc66a1d5d8bca', 'request_id': 'req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:44.730173 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9fcd161fc74a9891adc66a1d5d8bca', 'request_id': 'req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:44.735953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9fcd161fc74a9891adc66a1d5d8bca', 'request_id': 'req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:44.747226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] The token cache is being invalidate because project 7f9fcd161fc74a9891adc66a1d5d8bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:21:44.747523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:44.748196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:21:44.749738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1354/2745] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:21:44 2022] DELETE /identity/v3/projects/7f9fcd161fc74a9891adc66a1d5d8bca => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:21:54.075399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=16 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:21:54.743935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:02.997212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:02.998742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:03.023297 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320927e3-5027-49b1-9583-568567592df8 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:03.076266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:03.076971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:03.077263 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:03.077542 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:03.079672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:03.148898 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:03.150352 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:03.160636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1392/2746] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:22:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:22:12.874997 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:12.876681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:12.956210 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51decda-dce8-4eb9-8e7e-58c6d8a54b53 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:12.961925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:12.962560 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:12.962928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:12.963104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] PATH_INFO: `/v3/users/82a7db7a0f544b3eb9487bf00adbb742` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:12.964369 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a7db7a0f544b3eb9487bf00adbb742)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:12.966272 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:13.020490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a7db7a0f544b3eb9487bf00adbb742'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.034151 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a7db7a0f544b3eb9487bf00adbb742'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.034913 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a7db7a0f544b3eb9487bf00adbb742'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.051140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a7db7a0f544b3eb9487bf00adbb742'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.062870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a00b55be-b417-476a-b425-2a8aaf784367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a7db7a0f544b3eb9487bf00adbb742'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.071559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1355/2747] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:22:12 2022] DELETE /identity/v3/users/82a7db7a0f544b3eb9487bf00adbb742 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:22:13.155919 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=50 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:13.265399 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:13.278146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e40e7d-b9c1-491e-b20d-8b2bf614b0eb service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:13.342054 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:13.342871 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:13.343282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:13.343655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] PATH_INFO: `/v3/projects/8840b41dd8364a8a976f04d4fe421b8e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:13.345265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8840b41dd8364a8a976f04d4fe421b8e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:13.347643 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:13.398496 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8840b41dd8364a8a976f04d4fe421b8e', 'request_id': 'req-3ec870a4-ec67-40c7-8489-cb75e93dea81'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.405135 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8840b41dd8364a8a976f04d4fe421b8e', 'request_id': 'req-3ec870a4-ec67-40c7-8489-cb75e93dea81'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.414555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8840b41dd8364a8a976f04d4fe421b8e', 'request_id': 'req-3ec870a4-ec67-40c7-8489-cb75e93dea81'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.426824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] The token cache is being invalidate because project 8840b41dd8364a8a976f04d4fe421b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:22:13.427120 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.427815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:13.429374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1393/2748] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:22:13 2022] DELETE /identity/v3/projects/8840b41dd8364a8a976f04d4fe421b8e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:22:23.066516 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:23.423596 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:35.096172 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:35.096745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:35.096745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:35.117038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] MFA Rules not processed for user `6b069c6a7f1e4241816e5c91874cbcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:22:35.118284 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:35.175900 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1356/2749] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:22:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:22:43.652528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:43.654671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:43.718786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec870a4-ec67-40c7-8489-cb75e93dea81 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:43.724711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:43.725606 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:43.726099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:43.726587 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:43.729378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:43.796220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:43.797895 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:43.809039 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1394/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:22:45.168801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=25 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:50.823356 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:50.826153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:50.848930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9360dca-6c76-42c7-9beb-992147019ad0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:50.861716 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:50.864815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:50.868990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:50.868990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] PATH_INFO: `/v3/users/eee7bd9e4fa14b159277977043f4813c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:50.870299 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee7bd9e4fa14b159277977043f4813c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:50.872546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:50.929679 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee7bd9e4fa14b159277977043f4813c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:50.940096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee7bd9e4fa14b159277977043f4813c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:50.940790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee7bd9e4fa14b159277977043f4813c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:50.957604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee7bd9e4fa14b159277977043f4813c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:50.982966 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee7bd9e4fa14b159277977043f4813c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:50.991751 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1357/2751] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:22:50 2022] DELETE /identity/v3/users/eee7bd9e4fa14b159277977043f4813c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:22:51.181619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:51.193115 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737e7bf-3943-4c5d-8a0c-63c09e1b7337 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:51.257926 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:51.258589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:51.258938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:51.259216 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] PATH_INFO: `/v3/projects/b1331035c6654c0d9c8ba49f5a724f8c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:51.260452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1331035c6654c0d9c8ba49f5a724f8c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:51.262294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:51.314666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1331035c6654c0d9c8ba49f5a724f8c', 'request_id': 'req-03f9acc5-07d7-461c-8ad4-5d776bb32a44'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.316815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:51.323423 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1331035c6654c0d9c8ba49f5a724f8c', 'request_id': 'req-03f9acc5-07d7-461c-8ad4-5d776bb32a44'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.333378 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1331035c6654c0d9c8ba49f5a724f8c', 'request_id': 'req-03f9acc5-07d7-461c-8ad4-5d776bb32a44'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.353766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] The token cache is being invalidate because project b1331035c6654c0d9c8ba49f5a724f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:22:51.354016 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.354868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.356647 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1395/2752] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:22:51 2022] DELETE /identity/v3/projects/b1331035c6654c0d9c8ba49f5a724f8c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:22:51.361107 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:51.363022 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:51.378549 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0009d8-4c46-4a0c-9bec-c7a52fe99590 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:51.447851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:51.448828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:51.449275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:51.449485 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] PATH_INFO: `/v3/users/94ccc165e7754ecbb29bf6b9c854a568` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:51.450796 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ccc165e7754ecbb29bf6b9c854a568)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:51.452787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:51.508919 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ccc165e7754ecbb29bf6b9c854a568'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.516565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ccc165e7754ecbb29bf6b9c854a568'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.519816 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ccc165e7754ecbb29bf6b9c854a568'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.536477 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ccc165e7754ecbb29bf6b9c854a568'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.539368 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:22:51.545620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ccc165e7754ecbb29bf6b9c854a568'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.553253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1358/2753] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:22:51 2022] DELETE /identity/v3/users/94ccc165e7754ecbb29bf6b9c854a568 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:22:51.756798 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:22:51.758455 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:22:51.775507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9acc5-07d7-461c-8ad4-5d776bb32a44 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:22:51.839010 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:22:51.839687 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:22:51.839976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:22:51.840252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] PATH_INFO: `/v3/projects/e1b1f212ac024c8cab7ecf606e39179f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:22:51.841438 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b1f212ac024c8cab7ecf606e39179f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:22:51.843403 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:22:51.892660 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b1f212ac024c8cab7ecf606e39179f', 'request_id': 'req-28442481-de95-4137-b72f-0414444e0286'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.899061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b1f212ac024c8cab7ecf606e39179f', 'request_id': 'req-28442481-de95-4137-b72f-0414444e0286'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.909668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b1f212ac024c8cab7ecf606e39179f', 'request_id': 'req-28442481-de95-4137-b72f-0414444e0286'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.924522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] The token cache is being invalidate because project e1b1f212ac024c8cab7ecf606e39179f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:22:51.925001 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.925582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:22:51.927956 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1396/2754] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:22:51 2022] DELETE /identity/v3/projects/e1b1f212ac024c8cab7ecf606e39179f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:23:01.552522 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:23:01.919786 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:23:31.166181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:31.167663 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:31.224946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bac2e-17d9-4371-8363-a0aeb8b0e9eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:31.282798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:31.283474 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:31.283765 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:31.284040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:31.286310 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:31.397531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:31.399108 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4f1f1-ddeb-4d45-90bb-7e22214dd6d6 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:31.421490 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1359/2755] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:23:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:23:32.625392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:32.627723 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:32.644685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28442481-de95-4137-b72f-0414444e0286 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:32.648554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:32.649391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:32.649665 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:32.650233 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:32.652549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:32.669059 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:32.670588 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:32.687806 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1397/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:23:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:23:37.342676 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:37.343263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:37.343263 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:37.362491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] MFA Rules not processed for user `6b069c6a7f1e4241816e5c91874cbcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:23:37.363731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:37.391701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:23:37.426395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1360/2757] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:23:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:23:38.091324 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:38.093072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:38.168832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5bb0c-97be-41f3-b231-f1925b9b3b65 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:38.174851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:38.175528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:38.175813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:38.176090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:38.177885 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:38.190023 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:38.191696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:38.201828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1398/2758] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:23:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:23:46.179786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:46.190431 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b1e67-bd12-4d16-b005-1517b66b599d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:46.194147 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:46.194786 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:46.195100 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:46.195404 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:46.197291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:46.270648 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:46.272181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:46.284125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1361/2759] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:23:48.197528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=38 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:23:51.438442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:51.450260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76742f0b-6c2c-48f0-9b57-8c9f30434609 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:51.454386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:51.455086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:51.455531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:51.455822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:51.457791 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:51.468850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:51.470305 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:51.479719 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1399/2760] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:23:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:23:53.240190 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:53.241646 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:53.253951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d773c-27a1-4803-a38c-2c62abfb8602 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:53.259507 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:53.260253 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:53.260541 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:53.260821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] PATH_INFO: `/v3/users/d9cb42db2f9a4b80b17bef0a86945a6c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:53.262038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9cb42db2f9a4b80b17bef0a86945a6c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:53.264049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:53.314162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9cb42db2f9a4b80b17bef0a86945a6c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.322405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9cb42db2f9a4b80b17bef0a86945a6c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.323161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9cb42db2f9a4b80b17bef0a86945a6c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.335874 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9cb42db2f9a4b80b17bef0a86945a6c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.343049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9cb42db2f9a4b80b17bef0a86945a6c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.349125 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1362/2761] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:23:53 2022] DELETE /identity/v3/users/d9cb42db2f9a4b80b17bef0a86945a6c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:23:53.551118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:53.564421 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fa2ee-803e-4f28-9979-03de4299eeb2 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:53.626945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:53.627605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:53.627897 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:53.628182 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] PATH_INFO: `/v3/projects/d14de33f76ea44db8899900481299b87` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:53.629563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14de33f76ea44db8899900481299b87)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:53.632070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:53.686176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14de33f76ea44db8899900481299b87', 'request_id': 'req-c048955f-2f4e-4b7f-991c-eb514a63fc65'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.692613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14de33f76ea44db8899900481299b87', 'request_id': 'req-c048955f-2f4e-4b7f-991c-eb514a63fc65'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.702558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14de33f76ea44db8899900481299b87', 'request_id': 'req-c048955f-2f4e-4b7f-991c-eb514a63fc65'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.718994 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] The token cache is being invalidate because project d14de33f76ea44db8899900481299b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:23:53.719290 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.719999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.721637 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1400/2762] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:23:53 2022] DELETE /identity/v3/projects/d14de33f76ea44db8899900481299b87 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:23:53.725835 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:53.727640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:53.741385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdcf08-76c7-4177-b2b5-0a095c9e7afe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:53.789671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:23:53.807725 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:53.808377 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:53.808658 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:53.808933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] PATH_INFO: `/v3/users/fed72656b917472aa030ad4c8ba66d2b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:53.810209 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed72656b917472aa030ad4c8ba66d2b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:53.812193 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:53.871375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed72656b917472aa030ad4c8ba66d2b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.882332 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed72656b917472aa030ad4c8ba66d2b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.883067 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed72656b917472aa030ad4c8ba66d2b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.895384 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed72656b917472aa030ad4c8ba66d2b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.902726 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed72656b917472aa030ad4c8ba66d2b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:53.908849 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1363/2763] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:23:53 2022] DELETE /identity/v3/users/fed72656b917472aa030ad4c8ba66d2b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:23:54.125429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:23:54.127072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:23:54.140658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048955f-2f4e-4b7f-991c-eb514a63fc65 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:23:54.202766 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:23:54.203464 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:23:54.203750 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:23:54.204036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] PATH_INFO: `/v3/projects/5b33bcc707974091b1098e10867c5ccb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:23:54.205230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b33bcc707974091b1098e10867c5ccb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:23:54.207193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:23:54.230312 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:23:54.257986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b33bcc707974091b1098e10867c5ccb', 'request_id': 'req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:54.264351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b33bcc707974091b1098e10867c5ccb', 'request_id': 'req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:54.274064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b33bcc707974091b1098e10867c5ccb', 'request_id': 'req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:54.285538 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] The token cache is being invalidate because project 5b33bcc707974091b1098e10867c5ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:23:54.285920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:54.286671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5be6b1d7-9a1f-4ddc-9d04-6e0fc1151e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:23:54.288392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1401/2764] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:23:54 2022] DELETE /identity/v3/projects/5b33bcc707974091b1098e10867c5ccb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:24:03.905538 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:24:04.282210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:24:35.056281 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:24:35.058145 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:35.151591 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3631-c051-43cb-9122-66737f221710 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:24:35.262378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:24:35.263287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:24:35.263865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:24:35.264282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:24:35.266645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:35.282681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:24:35.284821 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:24:35.299365 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1364/2765] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:24:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:24:38.773973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:24:38.774528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:24:38.774528 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:24:38.794142 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] MFA Rules not processed for user `6b069c6a7f1e4241816e5c91874cbcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:24:38.795386 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:38.852555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1402/2766] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:24:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:24:39.199948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:24:39.201741 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:39.262048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5d999-36f4-414a-b288-e134b7075ccf service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:24:39.266117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:24:39.266737 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:24:39.267049 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:24:39.267311 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:24:39.269244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:39.279664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:24:39.281097 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:24:39.289142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1365/2767] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:24:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:24:48.845252 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=25 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:24:49.285762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:24:50.438541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:24:50.440189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:50.457195 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92345bf4-8ed4-472a-9d19-815718df3693 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:24:50.464047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f2f77d4f4c44890ac1ab88ff66100bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:24:50.464725 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:24:50.465007 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:24:50.465275 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:24:50.467178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:24:50.479965 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:24:50.481573 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:24:50.493022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1403/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:24:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:25:00.490602 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:43.949110 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:43.952632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:44.039717 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.041228 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:44.062417 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087024e-02cc-4dde-b7b2-8d2a9c16da84 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.067731 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.068366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.068652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.068931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.070718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:44.082530 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.084156 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.094062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1366/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:25:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:25:44.109625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f89fb-3039-43b4-9b87-6e066ac38d65 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.114286 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.114977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.115267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.115547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] PATH_INFO: `/v3/users/6b069c6a7f1e4241816e5c91874cbcd6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.116730 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b069c6a7f1e4241816e5c91874cbcd6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.118758 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.171651 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b069c6a7f1e4241816e5c91874cbcd6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.172320 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b069c6a7f1e4241816e5c91874cbcd6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.172923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b069c6a7f1e4241816e5c91874cbcd6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.182682 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b069c6a7f1e4241816e5c91874cbcd6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.195976 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e3ea0b6f-342c-48ae-b76e-09060afad8fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b069c6a7f1e4241816e5c91874cbcd6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.209609 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1404/2770] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:25:44 2022] DELETE /identity/v3/users/6b069c6a7f1e4241816e5c91874cbcd6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:25:44.390471 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.401632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622438d-ec46-443e-ba99-777ca27534ad service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.459098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.459801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.460085 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.460362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] PATH_INFO: `/v3/projects/adb91176f5f74f1199b2fc27c248cecd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.461534 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb91176f5f74f1199b2fc27c248cecd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.463496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.512854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb91176f5f74f1199b2fc27c248cecd', 'request_id': 'req-2bcf67db-a24b-4557-accb-1149d4e10032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.520444 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb91176f5f74f1199b2fc27c248cecd', 'request_id': 'req-2bcf67db-a24b-4557-accb-1149d4e10032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.525993 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb91176f5f74f1199b2fc27c248cecd', 'request_id': 'req-2bcf67db-a24b-4557-accb-1149d4e10032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.537756 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] The token cache is being invalidate because project adb91176f5f74f1199b2fc27c248cecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:25:44.538050 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.538822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2bcf67db-a24b-4557-accb-1149d4e10032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:44.540533 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1367/2771] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:25:44 2022] DELETE /identity/v3/projects/adb91176f5f74f1199b2fc27c248cecd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:44.546442 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45133681-cd83-4a81-a148-be07a245bebe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.546765 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45133681-cd83-4a81-a148-be07a245bebe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.547095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45133681-cd83-4a81-a148-be07a245bebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.567923 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-45133681-cd83-4a81-a148-be07a245bebe None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:44.569150 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45133681-cd83-4a81-a148-be07a245bebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:44.631971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1405/2772] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:44.640159 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.640451 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.640729 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.662851 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:44.664062 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:44.673126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1368/2773] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:44.677709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45133681-cd83-4a81-a148-be07a245bebe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.688990 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45133681-cd83-4a81-a148-be07a245bebe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.694611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.695329 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.695621 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.695892 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.696605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:25:44.697055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.698671 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.714744 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1406/2774] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:25:44.719616 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.729749 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f6096-c79e-496a-89ea-69a6aeef8408 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.735748 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.736385 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.736673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.736940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.737642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.739838 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.751864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:44.771529 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1369/2775] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:25:44 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:44.775849 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.786514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22fda-786b-4c39-85ba-a35d3d42eb14 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.790767 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:44.850625 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.851281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.851564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.851841 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.852507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.854422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.881532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1407/2776] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:25:44 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:44.883363 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.899205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceb1c6-428b-400d-88ea-7fdfdbcfaca7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.906202 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.906861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.907146 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.907424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.908133 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.909745 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.920140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1370/2777] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:44.924225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.935444 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309eade-22b0-42c3-ae76-103d34367ff9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.940547 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b49ab-d12b-4a5b-bf39-10c56a93a0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.941157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b49ab-d12b-4a5b-bf39-10c56a93a0af admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.941475 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b49ab-d12b-4a5b-bf39-10c56a93a0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.941759 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b49ab-d12b-4a5b-bf39-10c56a93a0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.942462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b49ab-d12b-4a5b-bf39-10c56a93a0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.944097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b49ab-d12b-4a5b-bf39-10c56a93a0af admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:44.955445 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1408/2778] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:44.959618 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:44.970086 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a817e0f-a4a3-4a75-914f-ae41c362cc76 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:44.975227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:44.975896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:44.976182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:44.976461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] PATH_INFO: `/v3/projects/827c4df3c4fa4c6c90d49d95b5383aad/users/736e223a106944b9bb943655cbe25d24/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:44.991425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=827c4df3c4fa4c6c90d49d95b5383aad, user_id=736e223a106944b9bb943655cbe25d24, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:44.994099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:45.008980 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1371/2779] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:25:44 2022] PUT /identity/v3/projects/827c4df3c4fa4c6c90d49d95b5383aad/users/736e223a106944b9bb943655cbe25d24/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:45.014090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dd9ac-59a8-46b1-ad5f-8da8ff65bf5c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:45.014380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dd9ac-59a8-46b1-ad5f-8da8ff65bf5c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:45.014675 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dd9ac-59a8-46b1-ad5f-8da8ff65bf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:45.042345 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fd1dd9ac-59a8-46b1-ad5f-8da8ff65bf5c None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:45.043497 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fd1dd9ac-59a8-46b1-ad5f-8da8ff65bf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:45.106862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1409/2780] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:45.120475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:45.123436 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:45.188493 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e8a5b-f440-4e20-a4b8-8622754fd1e7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:45.192865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:45.193495 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:45.193784 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:45.194064 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:45.204008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:45.205605 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:45.214977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1372/2781] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:25:45.865012 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:45.865333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:45.865640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:45.895289 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] MFA Rules not processed for user `736e223a106944b9bb943655cbe25d24`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:45.896525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:45.964124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1410/2782] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:25:45.968793 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:45.978929 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca2776-0d7d-4d36-a6ff-b9b1fc76027a service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:45.984505 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:45.985162 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:45.985511 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:45.985850 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:45.986531 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:45.988475 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:46.019359 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1373/2783] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:25:45 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:46.024181 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:46.036262 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e607ac5-3dcd-45e0-bcdd-19ec98512196 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:46.049555 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:46.097462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:46.098099 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:46.098388 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:46.098669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:46.099369 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:46.101291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:46.126776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1411/2784] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:25:46 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:46.128323 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:46.139130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f04a9-116f-4767-9480-8ae3c4b4db52 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:46.143922 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:46.144545 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:46.144836 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:46.145115 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:46.145873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:46.147504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:46.159610 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1374/2785] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 01:25:46.163989 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:46.174280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923fff3-7c9f-4d4a-9e80-efab33ed78cc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:46.179230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:46.179884 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:46.180191 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:46.180473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] PATH_INFO: `/v3/projects/595ecd12dca744079cf1e60fdcc8f03b/users/df21254f3ae942e3a6a58954abb4dac7/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:46.196078 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] RBAC: Authorizing `identity:create_grant(project_id=595ecd12dca744079cf1e60fdcc8f03b, user_id=df21254f3ae942e3a6a58954abb4dac7, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:46.198742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:46.214662 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1412/2786] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:25:46 2022] PUT /identity/v3/projects/595ecd12dca744079cf1e60fdcc8f03b/users/df21254f3ae942e3a6a58954abb4dac7/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:46.218889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:46.228960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e12989-ef26-4df1-a224-a4a2fb3b6b88 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:46.288154 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39634a9-a498-46e4-8074-ccadfb0aaf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:46.288876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39634a9-a498-46e4-8074-ccadfb0aaf00 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:46.289215 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39634a9-a498-46e4-8074-ccadfb0aaf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:46.289620 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39634a9-a498-46e4-8074-ccadfb0aaf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:46.290722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39634a9-a498-46e4-8074-ccadfb0aaf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:46.293250 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39634a9-a498-46e4-8074-ccadfb0aaf00 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:46.309053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1375/2787] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:46.313443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:46.324281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e7e1e-99ad-4f2e-a856-1ed4a698075a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:46.329854 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:46.330531 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:46.330822 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:46.331137 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] PATH_INFO: `/v3/projects/595ecd12dca744079cf1e60fdcc8f03b/users/df21254f3ae942e3a6a58954abb4dac7/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:46.333169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=595ecd12dca744079cf1e60fdcc8f03b, user_id=df21254f3ae942e3a6a58954abb4dac7, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:46.335876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:46.346370 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1413/2788] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:25:46 2022] PUT /identity/v3/projects/595ecd12dca744079cf1e60fdcc8f03b/users/df21254f3ae942e3a6a58954abb4dac7/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:47.007392 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:47.007961 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:47.008084 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:47.036319 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] MFA Rules not processed for user `df21254f3ae942e3a6a58954abb4dac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:47.037546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:47.046888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:47.109801 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1376/2789] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:48.345741 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:48.347794 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:48.399515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:48.427899 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8a881-e09a-4ca6-a232-fb1f13d65774 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:48.433163 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:48.433801 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:48.434090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:48.434459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:48.436339 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:48.451197 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:48.452814 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:48.463202 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1414/2790] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:25:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:25:48.475321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67d2d6-2b20-404e-84da-28c339640952 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:48.478329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:48.479024 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:48.479307 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:48.479584 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] PATH_INFO: `/v3/users/736e223a106944b9bb943655cbe25d24` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:48.480770 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=736e223a106944b9bb943655cbe25d24)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:48.483048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:48.533813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736e223a106944b9bb943655cbe25d24'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.542153 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736e223a106944b9bb943655cbe25d24'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.543037 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736e223a106944b9bb943655cbe25d24'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.557482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736e223a106944b9bb943655cbe25d24'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.564795 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736e223a106944b9bb943655cbe25d24'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.571040 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1377/2791] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:25:48 2022] DELETE /identity/v3/users/736e223a106944b9bb943655cbe25d24 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:48.753131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:48.764906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c963678-d6f1-4979-8883-8cd65a8b7876 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:48.820548 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:48.827582 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:48.828254 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:48.828552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:48.829086 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] PATH_INFO: `/v3/projects/827c4df3c4fa4c6c90d49d95b5383aad` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:48.830376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=827c4df3c4fa4c6c90d49d95b5383aad)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:48.832556 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:48.885227 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827c4df3c4fa4c6c90d49d95b5383aad', 'request_id': 'req-0d45e098-7108-45f0-828f-12a04d3073f8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.892658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827c4df3c4fa4c6c90d49d95b5383aad', 'request_id': 'req-0d45e098-7108-45f0-828f-12a04d3073f8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.902688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827c4df3c4fa4c6c90d49d95b5383aad', 'request_id': 'req-0d45e098-7108-45f0-828f-12a04d3073f8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.913800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] The token cache is being invalidate because project 827c4df3c4fa4c6c90d49d95b5383aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:25:48.914094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.914821 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:48.916505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1415/2792] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:25:48 2022] DELETE /identity/v3/projects/827c4df3c4fa4c6c90d49d95b5383aad => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:48.920789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:48.922290 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:48.935743 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43148032-62d3-4e86-944c-d1710a0f0d8b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:48.960644 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:48.996008 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:48.996653 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:48.996940 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:48.997218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] PATH_INFO: `/v3/users/df21254f3ae942e3a6a58954abb4dac7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:48.998459 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df21254f3ae942e3a6a58954abb4dac7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.000405 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.048792 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df21254f3ae942e3a6a58954abb4dac7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.057346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df21254f3ae942e3a6a58954abb4dac7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.057959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df21254f3ae942e3a6a58954abb4dac7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.070946 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df21254f3ae942e3a6a58954abb4dac7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.078423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0ea830cb-c193-4bd4-849d-02a9a4d81f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df21254f3ae942e3a6a58954abb4dac7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.084500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1378/2793] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:25:48 2022] DELETE /identity/v3/users/df21254f3ae942e3a6a58954abb4dac7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:25:49.272070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.273650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:49.287075 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e098-7108-45f0-828f-12a04d3073f8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.347764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.348440 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.348607 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.348882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] PATH_INFO: `/v3/projects/595ecd12dca744079cf1e60fdcc8f03b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.350132 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=595ecd12dca744079cf1e60fdcc8f03b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.352133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.401170 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '595ecd12dca744079cf1e60fdcc8f03b', 'request_id': 'req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.407436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '595ecd12dca744079cf1e60fdcc8f03b', 'request_id': 'req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.416724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '595ecd12dca744079cf1e60fdcc8f03b', 'request_id': 'req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.427583 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] The token cache is being invalidate because project 595ecd12dca744079cf1e60fdcc8f03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:25:49.427875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.428525 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-14c1d92d-aa2f-4c40-ab55-d2f4f71605cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:25:49.430087 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1416/2794] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:25:49 2022] DELETE /identity/v3/projects/595ecd12dca744079cf1e60fdcc8f03b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:25:49.435164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.435632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.435778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.455730 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:49.456951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:49.517960 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1379/2795] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:49.529688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.530021 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.530414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.555782 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:49.557019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:49.566396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1417/2796] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:49.570861 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.581361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a138-67a1-4ad6-b22e-906e31869e8b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.586992 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.587617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.587910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.588182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.588890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:25:49.589226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.590870 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.602819 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1380/2797] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:25:49.607095 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.616481 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a687afa-df18-4be8-81d8-14f509dbdd02 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.621589 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.622243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.622523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.622808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.623505 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.625407 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.635509 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:49.659431 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1418/2798] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:25:49 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:49.660647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.671305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119fc76-a01b-4eb1-9416-390b2674b112 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.703305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:49.732255 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.732878 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.733164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.733447 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.734257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.736236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.759329 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1381/2799] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:25:49 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:25:49.763616 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.774564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7d8f6-ae55-477b-835f-cc2dc664ccf6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.780240 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.780873 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.781167 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.781446 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.782165 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.783803 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.794124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1419/2800] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:49.798483 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.811634 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eba30d-93e3-446f-b51d-456da5c36e43 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.818915 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b5c7a-517c-41f1-80dc-666ef81e1361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.819822 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b5c7a-517c-41f1-80dc-666ef81e1361 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.820126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b5c7a-517c-41f1-80dc-666ef81e1361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.820423 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b5c7a-517c-41f1-80dc-666ef81e1361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.821425 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b5c7a-517c-41f1-80dc-666ef81e1361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.823953 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b5c7a-517c-41f1-80dc-666ef81e1361 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.834650 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1382/2801] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:49.839072 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:49.849783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d124cd-162d-4265-9417-b08046e88adc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:49.855709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:49.856394 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.856685 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.856964 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] PATH_INFO: `/v3/projects/74694341c3ed404684fbd56ba63d0508/users/efff176812d541f78986f92f39260b8f/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.872950 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=74694341c3ed404684fbd56ba63d0508, user_id=efff176812d541f78986f92f39260b8f, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:49.875659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:49.891991 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1420/2802] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:25:49 2022] PUT /identity/v3/projects/74694341c3ed404684fbd56ba63d0508/users/efff176812d541f78986f92f39260b8f/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:49.906446 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:49.906740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:49.907053 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:49.938164 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] MFA Rules not processed for user `efff176812d541f78986f92f39260b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:49.939642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:50.006643 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1383/2803] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:50.011389 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.077755 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b581b6-4fae-4b1a-968e-54baffe88ecb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.083657 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.084294 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.084579 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.084851 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.085523 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.087476 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.117422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1421/2804] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:25:50 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:50.121606 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.133544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a945a0-45b9-4dd1-8f78-51acd428d27f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.192843 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.193473 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.193761 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.194041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.194718 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.196680 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.218523 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1384/2805] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:25:50 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:25:50.222633 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.233459 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad61430-7280-40a5-bc5f-3634ae07d774 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.239037 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.239659 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.239949 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.240230 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.240938 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.242572 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.253343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1422/2806] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:50.257985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.268785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43dd5f-a73c-4033-9fe8-9546e9435b71 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.274185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.274868 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.275182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.275464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] PATH_INFO: `/v3/projects/1209222633454523a98b1e5a263d7f3a/users/914e2430f8294c29affd1f9ad9b59403/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.290252 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1209222633454523a98b1e5a263d7f3a, user_id=914e2430f8294c29affd1f9ad9b59403, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.292941 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.307071 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1385/2807] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:25:50 2022] PUT /identity/v3/projects/1209222633454523a98b1e5a263d7f3a/users/914e2430f8294c29affd1f9ad9b59403/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:50.311113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.321443 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6856c9a-2f3d-48ec-b3af-08d237fabc18 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.378907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c39d70-e955-484e-9061-86b42a2500e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.379546 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c39d70-e955-484e-9061-86b42a2500e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.379738 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c39d70-e955-484e-9061-86b42a2500e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.380391 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c39d70-e955-484e-9061-86b42a2500e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.381094 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c39d70-e955-484e-9061-86b42a2500e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.382693 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c39d70-e955-484e-9061-86b42a2500e1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.392500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1423/2808] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:25:50.396998 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.407925 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2d90-6834-4ab9-8970-2aa58db5a627 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.413352 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.414020 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.414282 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.414579 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] PATH_INFO: `/v3/projects/1209222633454523a98b1e5a263d7f3a/users/914e2430f8294c29affd1f9ad9b59403/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.416654 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1209222633454523a98b1e5a263d7f3a, user_id=914e2430f8294c29affd1f9ad9b59403, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.419419 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.421565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:50.432464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1386/2809] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:25:50 2022] PUT /identity/v3/projects/1209222633454523a98b1e5a263d7f3a/users/914e2430f8294c29affd1f9ad9b59403/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:25:50.502300 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.502591 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.502868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.537271 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] MFA Rules not processed for user `914e2430f8294c29affd1f9ad9b59403`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:25:50.538532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:50.553212 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:25:50.629564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1424/2810] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:25:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:25:50.641581 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.643665 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:25:50.702268 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc2c9b-d14f-49bd-a71a-4d25350f59f4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.706262 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.706924 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.707211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.707481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.716902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.718353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.728275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1387/2811] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:25:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:25:50.812146 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:25:50.824106 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a60d9-8601-44b0-8b72-51bb6ee8663e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:25:50.830266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f2576-4bdc-4d0e-b97e-144e13643f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:25:50.830955 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f2576-4bdc-4d0e-b97e-144e13643f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:25:50.831269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f2576-4bdc-4d0e-b97e-144e13643f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:25:50.831534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f2576-4bdc-4d0e-b97e-144e13643f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:25:50.898343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f2576-4bdc-4d0e-b97e-144e13643f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:25:50.899832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f2576-4bdc-4d0e-b97e-144e13643f20 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:25:50.911642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1425/2812] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:25:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:00.723927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:00.905987 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=51 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:30.107403 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:30.169832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e947-aebd-40cd-b539-2a58e662e395 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:30.174464 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:30.175138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:30.175420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:30.175696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] PATH_INFO: `/v3/users/efff176812d541f78986f92f39260b8f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:30.176892 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efff176812d541f78986f92f39260b8f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:30.178812 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:30.233696 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efff176812d541f78986f92f39260b8f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.242360 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efff176812d541f78986f92f39260b8f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.242959 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efff176812d541f78986f92f39260b8f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.255234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efff176812d541f78986f92f39260b8f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.262038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efff176812d541f78986f92f39260b8f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.268038 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1388/2813] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:30 2022] DELETE /identity/v3/users/efff176812d541f78986f92f39260b8f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:30.272831 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:30.273330 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:30.273536 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:30.293107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:30.294317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:30.354337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1426/2814] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:30.364362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:30.365865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:30.433488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc9dfc-63f5-462a-9757-c104393b3580 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:30.438673 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:30.439345 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:30.439631 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:30.439903 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:30.449526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:30.451021 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:30.460876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1389/2815] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:30.741861 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:30.743373 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:30.756047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba094-8858-48c3-a32c-8a90b713e5b1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:30.761575 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:30.762248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:30.762537 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:30.762819 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] PATH_INFO: `/v3/projects/74694341c3ed404684fbd56ba63d0508` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:30.764210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=74694341c3ed404684fbd56ba63d0508)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:30.767435 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:30.816284 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74694341c3ed404684fbd56ba63d0508', 'request_id': 'req-0e715aa7-1fed-4e89-b117-bba79904fbbb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.822336 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74694341c3ed404684fbd56ba63d0508', 'request_id': 'req-0e715aa7-1fed-4e89-b117-bba79904fbbb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.830876 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74694341c3ed404684fbd56ba63d0508', 'request_id': 'req-0e715aa7-1fed-4e89-b117-bba79904fbbb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.841437 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] The token cache is being invalidate because project 74694341c3ed404684fbd56ba63d0508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:30.841742 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.842494 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.844173 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1427/2816] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:30 2022] DELETE /identity/v3/projects/74694341c3ed404684fbd56ba63d0508 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:30.848455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:30.858479 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd3238-817e-4848-89da-dbca1d9beeff service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:30.877017 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:30.919782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:30.920544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:30.920834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:30.921114 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] PATH_INFO: `/v3/users/914e2430f8294c29affd1f9ad9b59403` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:30.922512 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914e2430f8294c29affd1f9ad9b59403)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:30.924487 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:30.971200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914e2430f8294c29affd1f9ad9b59403'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.980132 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914e2430f8294c29affd1f9ad9b59403'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:30.980798 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914e2430f8294c29affd1f9ad9b59403'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.000476 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914e2430f8294c29affd1f9ad9b59403'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.012565 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-32fac0c2-4db6-43c6-aa63-173140d21cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914e2430f8294c29affd1f9ad9b59403'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.023061 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1390/2817] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:30 2022] DELETE /identity/v3/users/914e2430f8294c29affd1f9ad9b59403 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:31.315380 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.316882 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:31.335714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715aa7-1fed-4e89-b117-bba79904fbbb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.393462 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.394041 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.394331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.394614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] PATH_INFO: `/v3/projects/1209222633454523a98b1e5a263d7f3a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.395881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1209222633454523a98b1e5a263d7f3a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.397811 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.430337 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:31.448815 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1209222633454523a98b1e5a263d7f3a', 'request_id': 'req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.455229 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1209222633454523a98b1e5a263d7f3a', 'request_id': 'req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.464367 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1209222633454523a98b1e5a263d7f3a', 'request_id': 'req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.475928 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] The token cache is being invalidate because project 1209222633454523a98b1e5a263d7f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:31.476193 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.477433 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d49e4ca0-96ab-4c61-b60d-ebd90bcba132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:31.478929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1428/2818] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:31 2022] DELETE /identity/v3/projects/1209222633454523a98b1e5a263d7f3a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:31.484265 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.484762 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.484871 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.505773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:31.507027 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:31.570564 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1391/2819] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:31.580480 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.580776 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.581055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.602468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:31.603813 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:31.612692 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1429/2820] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:31.617527 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.633129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de922968-fe31-4237-b53d-a945cbc06f62 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.641481 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.642113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.642395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.642672 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.643411 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:26:31.643785 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.645351 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.659889 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1392/2821] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:26:31.664514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.674936 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6845-400e-481e-8a4c-be34b4f75dee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.681015 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.681640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.681924 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.682211 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.682872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.684865 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.718071 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1430/2822] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:31 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:31.724169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.735502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7bcb6-96c6-4ee8-959d-4691e62fbe9d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.763099 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:31.799036 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.799664 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.799949 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.800233 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.800910 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.802873 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.825333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1393/2823] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:26:31 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:31.829808 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.840379 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe01eb3-52ab-4907-ae5f-cef070fc95a5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.845333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.845973 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.846255 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.846541 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.847287 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.848886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.859169 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1431/2824] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:31.863607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.877999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fab1f-4c76-4479-a37e-b92251de1512 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.883928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680223bb-41bb-4a2c-9425-35ad4e70de58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.884537 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680223bb-41bb-4a2c-9425-35ad4e70de58 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.884818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680223bb-41bb-4a2c-9425-35ad4e70de58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.885089 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680223bb-41bb-4a2c-9425-35ad4e70de58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.885789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680223bb-41bb-4a2c-9425-35ad4e70de58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.887426 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680223bb-41bb-4a2c-9425-35ad4e70de58 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.897995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1394/2825] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:31.902011 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:31.913313 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3381886-34b5-4b4d-aa1a-07b77e170a97 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:31.921667 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:31.922611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.923061 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.923395 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] PATH_INFO: `/v3/projects/ffa0c52f7c27466cb70785d62a2e2806/users/f8d39bcaec604ee98e4ee4af7afc3450/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:31.944200 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa0c52f7c27466cb70785d62a2e2806, user_id=f8d39bcaec604ee98e4ee4af7afc3450, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:31.946891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:31.961392 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1432/2826] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:31 2022] PUT /identity/v3/projects/ffa0c52f7c27466cb70785d62a2e2806/users/f8d39bcaec604ee98e4ee4af7afc3450/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:31.972790 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:31.973182 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:31.973502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.004251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] MFA Rules not processed for user `f8d39bcaec604ee98e4ee4af7afc3450`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:32.005837 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:32.073544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1395/2827] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:32.079946 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.146756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750de13d-9d41-4626-89a8-eff89a73dc50 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.152113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.152648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.152902 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.153184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.153869 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.155807 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.185796 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1433/2828] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:32 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:32.190041 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.201077 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304bcee-7898-4ece-bfdd-1a773e057bc8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.258069 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.258692 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.259006 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.259293 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.259952 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.261876 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.285421 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1396/2829] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:26:32 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:32.289600 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.300327 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bb742-d226-407a-affc-8571036235fa admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.305208 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.305852 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.306136 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.306434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.307166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.308745 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.319540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1434/2830] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:32.323719 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.334902 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65a905-0650-4ef8-a201-073c2b1bac8c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.339687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.340357 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.340630 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.340911 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] PATH_INFO: `/v3/projects/ea322686e9234ca5a4712f03bac4be1e/users/e3d35ef0d2c0495dabc165553494eed7/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.357467 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea322686e9234ca5a4712f03bac4be1e, user_id=e3d35ef0d2c0495dabc165553494eed7, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.360617 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.374778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1397/2831] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:32 2022] PUT /identity/v3/projects/ea322686e9234ca5a4712f03bac4be1e/users/e3d35ef0d2c0495dabc165553494eed7/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:32.378893 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.389468 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344557ec-c6dd-4617-bd31-3f9b6717e6ca admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.411184 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:32.452013 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578d8a5-dc11-450e-a2b8-a4f35f195a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.452658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578d8a5-dc11-450e-a2b8-a4f35f195a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.452947 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578d8a5-dc11-450e-a2b8-a4f35f195a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.453258 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578d8a5-dc11-450e-a2b8-a4f35f195a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.453971 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578d8a5-dc11-450e-a2b8-a4f35f195a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.455674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578d8a5-dc11-450e-a2b8-a4f35f195a50 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.466189 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1435/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:32.470711 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.476407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:32.483738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2104f2-30fe-4fb4-8684-ec84c5412b32 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.490015 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.491228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.491746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.492227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] PATH_INFO: `/v3/projects/ea322686e9234ca5a4712f03bac4be1e/users/e3d35ef0d2c0495dabc165553494eed7/roles/132431a386d641a8aaeba0496b601812` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.495275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea322686e9234ca5a4712f03bac4be1e, user_id=e3d35ef0d2c0495dabc165553494eed7, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.498535 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.512834 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1398/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:32 2022] PUT /identity/v3/projects/ea322686e9234ca5a4712f03bac4be1e/users/e3d35ef0d2c0495dabc165553494eed7/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:32.523753 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.524047 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.524331 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.554267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] MFA Rules not processed for user `e3d35ef0d2c0495dabc165553494eed7`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:32.555520 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:32.632577 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1436/2834] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:32.645104 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:32.647029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:32.711130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94948-aa4c-41ef-aac2-b41c091f8a90 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:32.715521 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:32.716142 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:32.716427 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:32.716704 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:32.726073 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:32.727544 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:32.736951 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1399/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 01:26:33.090580 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:33.100141 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf95004-39b0-4aaf-b02a-1f9343270cab None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:33.105479 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d35ef0d2c0495dabc165553494eed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea322686e9234ca5a4712f03bac4be1e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:33.106124 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:33.106426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:33.106701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:33.107467 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:26:33.107774 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:33.109350 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:33.121855 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1437/2836] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:26:33.126500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:33.136325 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd40707-7a0d-4aed-8268-6cfda77f0826 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:33.142383 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d35ef0d2c0495dabc165553494eed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea322686e9234ca5a4712f03bac4be1e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:33.143043 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:33.143335 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:33.143626 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:33.144304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:33.146211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:33.176083 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1400/2837] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:33 2022] POST /identity/v3/projects => generated 353 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:33.180317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:33.190642 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051e2b-e557-4d50-ac45-baf08007d806 tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:33.253030 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d35ef0d2c0495dabc165553494eed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea322686e9234ca5a4712f03bac4be1e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:33.253663 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:33.253939 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:33.254226 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:33.255374 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:26:33.255650 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:33.257267 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:33.269862 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1438/2838] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:26:33.310917 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:33.328710 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7339f7-0739-4683-ab37-053991779f2d tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:33.334995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d35ef0d2c0495dabc165553494eed7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea322686e9234ca5a4712f03bac4be1e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:33.335651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:33.335938 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:33.336212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] PATH_INFO: `/v3/projects/09a1e27fe3b34a479c368cf632e2ed30` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:33.337542 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorizing `identity:delete_project(project_id=09a1e27fe3b34a479c368cf632e2ed30)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:33.339499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:33.389740 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a1e27fe3b34a479c368cf632e2ed30', 'request_id': 'req-c48c42d3-d316-433f-815c-c739a8c20f9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.398769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a1e27fe3b34a479c368cf632e2ed30', 'request_id': 'req-c48c42d3-d316-433f-815c-c739a8c20f9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.404742 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a1e27fe3b34a479c368cf632e2ed30', 'request_id': 'req-c48c42d3-d316-433f-815c-c739a8c20f9e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.417048 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] The token cache is being invalidate because project 09a1e27fe3b34a479c368cf632e2ed30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:33.417305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.418090 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c48c42d3-d316-433f-815c-c739a8c20f9e tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.419797 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1401/2839] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:33 2022] DELETE /identity/v3/projects/09a1e27fe3b34a479c368cf632e2ed30 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:33.621654 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:33.623584 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:33.654785 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:33.693157 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e11e1-34fa-416d-909c-dd0fb61f131a tempest-VolumeQuotaClassesTest-1918044205 tempest-VolumeQuotaClassesTest-1918044205-project-admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:33.751769 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:33.752425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:33.752711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:33.752993 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] PATH_INFO: `/v3/users/f8d39bcaec604ee98e4ee4af7afc3450` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:33.754198 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d39bcaec604ee98e4ee4af7afc3450)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:33.756514 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:33.805425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d39bcaec604ee98e4ee4af7afc3450'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.806178 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d39bcaec604ee98e4ee4af7afc3450'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.806736 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d39bcaec604ee98e4ee4af7afc3450'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.813843 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d39bcaec604ee98e4ee4af7afc3450'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.825906 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d39bcaec604ee98e4ee4af7afc3450'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:33.839877 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1439/2840] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:33 2022] DELETE /identity/v3/users/f8d39bcaec604ee98e4ee4af7afc3450 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:33.844445 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:33.844981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:33.844981 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:33.865424 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:33.866636 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:33.920056 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:33.935140 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1402/2841] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:33.945055 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:33.946619 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:34.008168 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fad6f-5a61-40e7-b85c-8a736ab4e925 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:34.012674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:34.013298 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:34.013605 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:34.013881 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:34.023986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:34.026268 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:34.027740 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:34.037934 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1440/2842] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:34.315519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:34.317342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:34.337318 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db646b-a00b-4286-af86-fdf18ce851b8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:34.344117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:34.348420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:34.348420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:34.348420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] PATH_INFO: `/v3/projects/ffa0c52f7c27466cb70785d62a2e2806` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:34.348420 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa0c52f7c27466cb70785d62a2e2806)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:34.349494 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:34.400399 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa0c52f7c27466cb70785d62a2e2806', 'request_id': 'req-85b22dd1-592a-452f-9500-7f6919d34d7d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.408070 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa0c52f7c27466cb70785d62a2e2806', 'request_id': 'req-85b22dd1-592a-452f-9500-7f6919d34d7d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.413713 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa0c52f7c27466cb70785d62a2e2806', 'request_id': 'req-85b22dd1-592a-452f-9500-7f6919d34d7d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.424973 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] The token cache is being invalidate because project ffa0c52f7c27466cb70785d62a2e2806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:34.425266 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.425990 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.427642 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1403/2843] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:34 2022] DELETE /identity/v3/projects/ffa0c52f7c27466cb70785d62a2e2806 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:34.431932 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:34.442256 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a5702-d8d6-4a9d-aa88-19fcd789e346 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:34.506868 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:34.507532 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:34.507832 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:34.508107 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] PATH_INFO: `/v3/users/e3d35ef0d2c0495dabc165553494eed7` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:34.509311 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d35ef0d2c0495dabc165553494eed7)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:34.511261 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:34.559688 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d35ef0d2c0495dabc165553494eed7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.560343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d35ef0d2c0495dabc165553494eed7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.560992 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d35ef0d2c0495dabc165553494eed7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.568118 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d35ef0d2c0495dabc165553494eed7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.580291 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e8622a91-ec5f-48e4-8916-ac4ac3acd3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d35ef0d2c0495dabc165553494eed7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:34.595974 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1441/2844] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:34 2022] DELETE /identity/v3/users/e3d35ef0d2c0495dabc165553494eed7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:26:34.878251 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:34.880200 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:34.893526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b22dd1-592a-452f-9500-7f6919d34d7d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:34.950684 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:34.951366 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:34.951651 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:34.951935 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] PATH_INFO: `/v3/projects/ea322686e9234ca5a4712f03bac4be1e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:34.953169 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea322686e9234ca5a4712f03bac4be1e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:34.955136 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.002337 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea322686e9234ca5a4712f03bac4be1e', 'request_id': 'req-936e8649-7465-4f02-8a69-8c447aa668ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:35.013105 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea322686e9234ca5a4712f03bac4be1e', 'request_id': 'req-936e8649-7465-4f02-8a69-8c447aa668ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:35.021408 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea322686e9234ca5a4712f03bac4be1e', 'request_id': 'req-936e8649-7465-4f02-8a69-8c447aa668ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:35.036942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] The token cache is being invalidate because project ea322686e9234ca5a4712f03bac4be1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:35.037234 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:35.038029 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-936e8649-7465-4f02-8a69-8c447aa668ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:35.039738 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1404/2845] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:34 2022] DELETE /identity/v3/projects/ea322686e9234ca5a4712f03bac4be1e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:35.045905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.046419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.046419 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.066486 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:35.067707 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:35.130301 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1442/2846] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:35.139723 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.140013 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.140291 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.155401 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:35.161261 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:35.163165 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:35.172361 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1405/2847] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:26:35.176820 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.186891 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c38eba-6876-4bc0-8a96-b9d849d5c7b5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.192265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.192886 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.193166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.193434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.194133 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:26:35.194469 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.196122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.208674 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1443/2848] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:26:35.212995 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.222453 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43f3a1-f8bf-4298-b32b-ea2208027f55 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.227693 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.228289 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.228575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.228854 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.229540 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.231499 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.261510 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1406/2849] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:35.265810 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.271223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:35.277691 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b01e-e6d6-4c32-bdcc-886e68e5bb61 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.338762 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.339426 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.339715 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.339980 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.340716 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.342648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.366053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1444/2850] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:35.370304 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.381559 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a865b24-aed7-4f37-bf3d-373eb7fe22e9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.387305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.387927 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.388207 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.388478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.389177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.390887 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.401303 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1407/2851] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:35.405322 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.415959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da803c-131a-48f8-9aa0-683a9320ba3c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.421610 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6da739-21d1-4fc1-a7e1-695125413bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.422342 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da739-21d1-4fc1-a7e1-695125413bda admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.422666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da739-21d1-4fc1-a7e1-695125413bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.422999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da739-21d1-4fc1-a7e1-695125413bda admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.423711 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da739-21d1-4fc1-a7e1-695125413bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.425319 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da739-21d1-4fc1-a7e1-695125413bda admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.436368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1445/2852] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:35.440557 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.450934 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc7837-fa1b-4e42-945f-50f2810ed238 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.456308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.456991 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.457308 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.457595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] PATH_INFO: `/v3/projects/0a0e9c32ab0142f09f5799469fcdbf7f/users/7a12c83606bb4cc2b88a928d9f5f379a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.472944 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0e9c32ab0142f09f5799469fcdbf7f, user_id=7a12c83606bb4cc2b88a928d9f5f379a, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.475641 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.490461 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1408/2853] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:35 2022] PUT /identity/v3/projects/0a0e9c32ab0142f09f5799469fcdbf7f/users/7a12c83606bb4cc2b88a928d9f5f379a/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:35.501317 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.501611 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.501889 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.534269 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] MFA Rules not processed for user `7a12c83606bb4cc2b88a928d9f5f379a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:35.535617 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:35.599912 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1446/2854] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:35.607218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.677230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31beaf7-b426-487c-b55a-2754651e482e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.682212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.684638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.684638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.684638 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.684965 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.686694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.716227 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1409/2855] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:35.720666 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.732036 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41c90b-9fee-4e3d-bbb6-5daaa73d7ce6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.791357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.791977 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.792260 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.792533 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.793196 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.795134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.817250 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:35.825070 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1447/2856] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:26:35 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:35.828928 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.839782 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706f547-0281-4436-a150-0ab086560706 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.845491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.846111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.846395 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.846669 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.847402 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.849010 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.859815 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1410/2857] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:35.863972 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.874176 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13617-2763-48db-92c3-c55babbc441a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.879668 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.880697 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.881019 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.881265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] PATH_INFO: `/v3/projects/39d8bba053ca438384ae625dac577627/users/8a91b9dd67854c20a0aaf3925956522e/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.897053 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d8bba053ca438384ae625dac577627, user_id=8a91b9dd67854c20a0aaf3925956522e, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.899756 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:35.914875 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1448/2858] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:35 2022] PUT /identity/v3/projects/39d8bba053ca438384ae625dac577627/users/8a91b9dd67854c20a0aaf3925956522e/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:26:35.919032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:35.929656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff917e4-ea7a-45b7-bf61-120f78558013 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:35.986372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f87a1-7617-49a6-ab46-395c7d74928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:35.987028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f87a1-7617-49a6-ab46-395c7d74928e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:35.987375 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f87a1-7617-49a6-ab46-395c7d74928e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:35.987645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f87a1-7617-49a6-ab46-395c7d74928e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:35.988353 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f87a1-7617-49a6-ab46-395c7d74928e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:35.989958 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f87a1-7617-49a6-ab46-395c7d74928e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:36.004406 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1411/2859] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:36.009077 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:36.020562 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6d6c1-0b19-4f0e-8cfd-d8e233b91e80 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:36.026558 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:36.027281 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:36.027570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:36.027850 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] PATH_INFO: `/v3/projects/39d8bba053ca438384ae625dac577627/users/8a91b9dd67854c20a0aaf3925956522e/roles/132431a386d641a8aaeba0496b601812` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:36.030138 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d8bba053ca438384ae625dac577627, user_id=8a91b9dd67854c20a0aaf3925956522e, role_id=132431a386d641a8aaeba0496b601812)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:36.032848 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:36.044264 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1449/2860] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:36 2022] PUT /identity/v3/projects/39d8bba053ca438384ae625dac577627/users/8a91b9dd67854c20a0aaf3925956522e/roles/132431a386d641a8aaeba0496b601812 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:36.054907 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:36.055205 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:36.055491 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:36.082111 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:36.085687 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] MFA Rules not processed for user `8a91b9dd67854c20a0aaf3925956522e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:36.087645 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:36.159469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1412/2861] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:36.171122 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:36.173028 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:36.234338 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426de9c2-6a91-4bf9-b498-8b770ef0896d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:36.238303 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:36.238951 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:36.239238 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:36.239518 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:36.248552 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:36.249986 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:36.259724 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1450/2862] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:36.342647 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:36.352773 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20f0de-a558-4a7c-b8fa-f89417cc0230 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:36.356715 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da847b75-5fc8-4863-8a5b-f1f7b06fba4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:36.357334 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da847b75-5fc8-4863-8a5b-f1f7b06fba4a service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:36.357615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da847b75-5fc8-4863-8a5b-f1f7b06fba4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:36.357893 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da847b75-5fc8-4863-8a5b-f1f7b06fba4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:36.417955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da847b75-5fc8-4863-8a5b-f1f7b06fba4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:36.419463 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da847b75-5fc8-4863-8a5b-f1f7b06fba4a service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:36.428520 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1413/2863] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:36.711302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:36.780564 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0d42a-1aca-4376-849d-dc4a98e92673 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:36.785988 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:36.786867 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:36.787225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:36.787472 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] PATH_INFO: `/v3/users/7a12c83606bb4cc2b88a928d9f5f379a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:36.788696 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a12c83606bb4cc2b88a928d9f5f379a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:36.790614 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:36.851679 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a12c83606bb4cc2b88a928d9f5f379a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:36.852714 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a12c83606bb4cc2b88a928d9f5f379a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:36.853770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a12c83606bb4cc2b88a928d9f5f379a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:36.864266 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a12c83606bb4cc2b88a928d9f5f379a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:36.872636 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:36.878764 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a12c83606bb4cc2b88a928d9f5f379a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:36.894959 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1451/2864] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:36 2022] DELETE /identity/v3/users/7a12c83606bb4cc2b88a928d9f5f379a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:36.899760 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:36.900057 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:36.900346 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:36.919999 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:36.921212 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:36.981789 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1414/2865] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:26:36.992456 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:36.995080 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:37.068729 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e536d65-8b47-48a2-9462-b06fe2173755 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:37.072731 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:37.073368 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:37.073648 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:37.073920 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:37.083578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:37.085017 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:37.094999 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1452/2866] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:37.394126 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:37.395709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:37.407754 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c724f-dc54-4bc8-9053-0f504bc66793 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:37.413244 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:37.413895 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:37.414184 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:37.414462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] PATH_INFO: `/v3/projects/0a0e9c32ab0142f09f5799469fcdbf7f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:37.415814 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0e9c32ab0142f09f5799469fcdbf7f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:37.418181 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:37.455937 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:37.468287 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0e9c32ab0142f09f5799469fcdbf7f', 'request_id': 'req-614de0d2-6e8f-49ad-913c-3473473df8f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.478943 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0e9c32ab0142f09f5799469fcdbf7f', 'request_id': 'req-614de0d2-6e8f-49ad-913c-3473473df8f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.484853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0e9c32ab0142f09f5799469fcdbf7f', 'request_id': 'req-614de0d2-6e8f-49ad-913c-3473473df8f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.496598 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] The token cache is being invalidate because project 0a0e9c32ab0142f09f5799469fcdbf7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:37.496890 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.497569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.499236 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1415/2867] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:37 2022] DELETE /identity/v3/projects/0a0e9c32ab0142f09f5799469fcdbf7f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:37.503576 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:37.514292 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d96c32-2be8-4927-8a27-3e0aa26eb0e8 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:37.581694 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:37.582265 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:37.582554 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:37.582828 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] PATH_INFO: `/v3/users/8a91b9dd67854c20a0aaf3925956522e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:37.584140 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a91b9dd67854c20a0aaf3925956522e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:37.586102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:37.635235 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a91b9dd67854c20a0aaf3925956522e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.635905 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a91b9dd67854c20a0aaf3925956522e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.636540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a91b9dd67854c20a0aaf3925956522e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.643452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a91b9dd67854c20a0aaf3925956522e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.657954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0b294b77-ccde-42fb-b834-cc3d5d716f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a91b9dd67854c20a0aaf3925956522e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:37.672640 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1453/2868] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:37 2022] DELETE /identity/v3/users/8a91b9dd67854c20a0aaf3925956522e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:37.962865 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:37.964502 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:37.978218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614de0d2-6e8f-49ad-913c-3473473df8f2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.037923 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.038640 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.038955 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.039228 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] PATH_INFO: `/v3/projects/39d8bba053ca438384ae625dac577627` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.040519 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d8bba053ca438384ae625dac577627)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.042455 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.090374 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d8bba053ca438384ae625dac577627', 'request_id': 'req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:38.098681 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d8bba053ca438384ae625dac577627', 'request_id': 'req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:38.109130 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d8bba053ca438384ae625dac577627', 'request_id': 'req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:38.122759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] The token cache is being invalidate because project 39d8bba053ca438384ae625dac577627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:38.123196 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:38.123948 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d0ff43e8-bd87-4b47-915e-5c6e11ab2bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:38.125574 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1416/2869] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:37 2022] DELETE /identity/v3/projects/39d8bba053ca438384ae625dac577627 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:38.131126 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.131429 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.131712 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.146253 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:38.153473 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:38.155414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:38.216117 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1454/2870] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:38.224547 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.224916 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.225211 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.245144 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:38.246362 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:38.255456 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1417/2871] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:38.259722 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.269658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaddf26-dc1f-42a8-9276-ec5383e43718 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.275057 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.275678 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.275963 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.276239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.276929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:26:38.277236 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.278824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.290357 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1455/2872] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 01:26:38.294525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.304257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53bf1-9346-443f-b220-27795dd86d01 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.310321 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.310978 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.311267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.311543 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.312199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.314117 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.346065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1418/2873] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:38 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:38.352029 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.362102 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffabf68-f820-47bc-9a46-d6383b20be2e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.421064 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.421699 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.421970 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.422248 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.422935 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.424858 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.451360 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1456/2874] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:26:38 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:38.452575 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.463496 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a07624-a078-40e0-9a54-5ec3ad42e9ad admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.468982 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.469602 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.469886 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.470161 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.470885 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.472515 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.483113 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1419/2875] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:38.487578 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.498655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbb53a-a54b-4087-8f2c-b4c330f6c833 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.504466 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58121511-3cb2-4375-b10f-e52ade98c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.505422 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58121511-3cb2-4375-b10f-e52ade98c718 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.505701 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58121511-3cb2-4375-b10f-e52ade98c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.505967 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58121511-3cb2-4375-b10f-e52ade98c718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.506652 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58121511-3cb2-4375-b10f-e52ade98c718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.508383 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58121511-3cb2-4375-b10f-e52ade98c718 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.519613 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1457/2876] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:38.524573 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.538639 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00207d73-f977-4db6-88d9-e9d7478086a3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.544219 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.544901 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.545185 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.545462 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] PATH_INFO: `/v3/projects/3db56edf7bd145c08695c893d0d38d85/users/6262b9a5a734499496f0ff677e6077db/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.554407 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:38.562746 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db56edf7bd145c08695c893d0d38d85, user_id=6262b9a5a734499496f0ff677e6077db, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.565994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.579478 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1420/2877] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:38 2022] PUT /identity/v3/projects/3db56edf7bd145c08695c893d0d38d85/users/6262b9a5a734499496f0ff677e6077db/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:38.590024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.590333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.590608 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.619771 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] MFA Rules not processed for user `6262b9a5a734499496f0ff677e6077db`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:38.620916 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:38.690829 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1458/2878] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:38.710504 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:38.713065 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:38.775840 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e78c00-1031-426d-a004-3a42da66a09b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:38.780372 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6146fcd207164df18b9c08e8fcc1adb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:38.780994 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:38.781276 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:38.781546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:38.790709 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:38.792197 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:38.801226 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1421/2879] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:43.782702 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:43.784909 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:43.791353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:43.859737 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235f91a-ef85-44e0-8e2e-700440a85647 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:43.865376 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:43.866014 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:43.866280 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:43.866549 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:43.868512 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:43.880485 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:43.882111 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a176b-3b0c-4a5d-a2ba-3fb89a993c32 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:43.892123 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1459/2880] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:48.797183 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:53.888220 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=31 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:56.626116 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:56.694267 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa2f0b-f53f-4922-818c-126782149a86 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:56.700787 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:56.701546 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:56.701828 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:56.702103 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] PATH_INFO: `/v3/users/6262b9a5a734499496f0ff677e6077db` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:56.703500 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6262b9a5a734499496f0ff677e6077db)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:56.705615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:56.755333 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6262b9a5a734499496f0ff677e6077db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:56.762823 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6262b9a5a734499496f0ff677e6077db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:56.763488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6262b9a5a734499496f0ff677e6077db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:56.775218 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6262b9a5a734499496f0ff677e6077db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:56.782652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6262b9a5a734499496f0ff677e6077db'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:56.790039 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1422/2881] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:26:56 2022] DELETE /identity/v3/users/6262b9a5a734499496f0ff677e6077db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:56.795045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:56.795551 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:56.795655 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:56.815090 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:56.816326 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:56.880507 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1460/2882] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 01:26:56.890701 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:56.892378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:56.957671 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c9553-94e5-4926-b565-c41ef309cbfb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:56.961656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e9299-8bb0-4839-aff9-de0e3a65da7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:56.962275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9299-8bb0-4839-aff9-de0e3a65da7e service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:56.962556 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9299-8bb0-4839-aff9-de0e3a65da7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:56.962864 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9299-8bb0-4839-aff9-de0e3a65da7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:56.972604 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e9299-8bb0-4839-aff9-de0e3a65da7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:56.974032 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e9299-8bb0-4839-aff9-de0e3a65da7e service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:56.983818 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1423/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:26:57.268134 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.269709 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:57.285302 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0babdd-da30-4143-b90d-9350c5491ce6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.290817 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.291410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.291700 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.291968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] PATH_INFO: `/v3/projects/3db56edf7bd145c08695c893d0d38d85` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.293384 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db56edf7bd145c08695c893d0d38d85)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.295335 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.344353 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db56edf7bd145c08695c893d0d38d85', 'request_id': 'req-18926b1f-3be3-448c-b373-5ae861b22405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:57.350460 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db56edf7bd145c08695c893d0d38d85', 'request_id': 'req-18926b1f-3be3-448c-b373-5ae861b22405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:57.359968 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db56edf7bd145c08695c893d0d38d85', 'request_id': 'req-18926b1f-3be3-448c-b373-5ae861b22405'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:57.370482 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] The token cache is being invalidate because project 3db56edf7bd145c08695c893d0d38d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:26:57.370770 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:57.371414 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-18926b1f-3be3-448c-b373-5ae861b22405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:26:57.372929 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1461/2884] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:26:57 2022] DELETE /identity/v3/projects/3db56edf7bd145c08695c893d0d38d85 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:57.377985 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.378275 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.378550 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.398931 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:57.400138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:57.417753 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:57.463526 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1424/2885] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:57.472534 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.472824 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.473097 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.493239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:57.494449 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:57.504239 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1462/2886] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:57.511002 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.527569 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23b263-49fc-40c2-969b-66a2d6d12a91 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.534259 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.534881 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.535189 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.535469 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.536177 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 01:26:57.536488 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.538096 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.551525 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1425/2887] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Jul 28 01:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 01:26:57.555954 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.565457 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad3b5f-6e53-4401-a918-7cef2b06405b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.570563 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.571223 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.571513 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.571783 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.572452 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.574351 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.604658 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1463/2888] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Jul 28 01:26:57 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 01:26:57.608832 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.619060 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21a0f-2037-4dfb-9834-2fc62d406f72 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.687586 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.688342 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.688448 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.688778 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.689484 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.691470 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.715656 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1426/2889] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jul 28 01:26:57 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 01:26:57.720272 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.731210 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd64e31-935d-4a64-aeb9-a6048994028a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.736434 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.737051 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.737358 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.737634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.738333 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.739957 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.749945 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1464/2890] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:57.753942 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.764245 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7f7b4-55f1-4616-a52e-df0519d7eb96 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.769192 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43f34-5d60-4059-a369-38567a58c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.769813 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43f34-5d60-4059-a369-38567a58c25d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.770101 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43f34-5d60-4059-a369-38567a58c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.770382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43f34-5d60-4059-a369-38567a58c25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.771122 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed43f34-5d60-4059-a369-38567a58c25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.772752 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed43f34-5d60-4059-a369-38567a58c25d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.783607 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1427/2891] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Jul 28 01:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 01:26:57.787713 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.798243 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd6023-93d1-4475-8a3c-075542284d12 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:57.803436 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:57.804110 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.804396 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.804672 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] PATH_INFO: `/v3/projects/01764af693e949a797eb85c4ddf4e76a/users/ac4701b7bd8c42079a52c95ecbe17422/roles/90d9b9386f5b4e0ebc5bba9c29b9d836` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.819872 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01764af693e949a797eb85c4ddf4e76a, user_id=ac4701b7bd8c42079a52c95ecbe17422, role_id=90d9b9386f5b4e0ebc5bba9c29b9d836)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:57.822500 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:57.837669 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1465/2892] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Jul 28 01:26:57 2022] PUT /identity/v3/projects/01764af693e949a797eb85c4ddf4e76a/users/ac4701b7bd8c42079a52c95ecbe17422/roles/90d9b9386f5b4e0ebc5bba9c29b9d836 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:26:57.853933 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:57.854322 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:57.854759 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:57.884811 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] MFA Rules not processed for user `ac4701b7bd8c42079a52c95ecbe17422`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:26:57.886051 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:57.952930 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1428/2893] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:26:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:26:57.965241 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:26:57.967113 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:26:57.993045 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:26:58.052282 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a50d28-d14e-4419-a37c-c416c0db4409 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:26:58.056540 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997b212-f01b-46ce-a84b-ec32d2b84ad2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c7dc51159c4fcd8ae520f82ef7ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:26:58.057166 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997b212-f01b-46ce-a84b-ec32d2b84ad2 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:26:58.057453 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997b212-f01b-46ce-a84b-ec32d2b84ad2 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:26:58.057734 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997b212-f01b-46ce-a84b-ec32d2b84ad2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:26:58.067516 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997b212-f01b-46ce-a84b-ec32d2b84ad2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:26:58.069131 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997b212-f01b-46ce-a84b-ec32d2b84ad2 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:26:58.079343 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1466/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:26:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:27:07.949769 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:27:08.074800 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=26 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:27:28.584722 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:27:28.650482 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2297ec1-fa87-49f5-b933-3aa21c9d09cd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:27:28.655652 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:27:28.656305 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:27:28.656585 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:27:28.656857 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] PATH_INFO: `/v3/users/ac4701b7bd8c42079a52c95ecbe17422` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:27:28.658129 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4701b7bd8c42079a52c95ecbe17422)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:27:28.660098 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:27:28.717611 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4701b7bd8c42079a52c95ecbe17422'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:28.726595 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4701b7bd8c42079a52c95ecbe17422'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:28.727230 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4701b7bd8c42079a52c95ecbe17422'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:28.740028 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4701b7bd8c42079a52c95ecbe17422'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:28.747175 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4701b7bd8c42079a52c95ecbe17422'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:28.753378 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1429/2895] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Jul 28 01:27:28 2022] DELETE /identity/v3/users/ac4701b7bd8c42079a52c95ecbe17422 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 01:27:28.758372 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:27:28.758907 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:27:28.759043 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:27:28.778277 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] MFA Rules not processed for user `52c6f46685584e8380771f008e5f1d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 01:27:28.779527 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:27:28.848634 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1467/2896] 10.222.0.17 () {58 vars in 958 bytes} [Thu Jul 28 01:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 01:27:28.861694 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:27:28.863393 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:27:28.924257 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc75c6-0438-4e67-89f3-1e2ba3c5fc39 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:27:28.928004 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fda0e3ee3624ac19de85bb3db30e2dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:27:28.928632 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:27:28.928891 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:27:28.929170 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:27:28.938454 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:27:28.939977 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:27:28.949772 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1430/2897] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:27:29.237365 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:27:29.239024 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:27:29.252022 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caac95a-8b3e-44aa-880b-e170e2b6479e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:27:29.257818 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52c6f46685584e8380771f008e5f1d77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb3fd2235b214e75887726b6b079e208', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:27:29.258522 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:27:29.258812 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:27:29.259128 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] PATH_INFO: `/v3/projects/01764af693e949a797eb85c4ddf4e76a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:27:29.260410 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01764af693e949a797eb85c4ddf4e76a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:27:29.262349 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:27:29.312139 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01764af693e949a797eb85c4ddf4e76a', 'request_id': 'req-9901b42d-857a-4e5c-b417-4d3d9055f469'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:29.318425 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01764af693e949a797eb85c4ddf4e76a', 'request_id': 'req-9901b42d-857a-4e5c-b417-4d3d9055f469'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:29.327418 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01764af693e949a797eb85c4ddf4e76a', 'request_id': 'req-9901b42d-857a-4e5c-b417-4d3d9055f469'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:29.338276 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] The token cache is being invalidate because project 01764af693e949a797eb85c4ddf4e76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 01:27:29.338570 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:29.339225 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9901b42d-857a-4e5c-b417-4d3d9055f469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 01:27:29.340779 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1468/2898] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Jul 28 01:27:29 2022] DELETE /identity/v3/projects/01764af693e949a797eb85c4ddf4e76a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 01:27:38.945583 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:27:39.335366 dv-cinder-815582-11-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 01:31:48.792888 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 01:31:48.794615 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:31:48.947199 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd62379-a500-44b5-b043-05a5dbb0dd14 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 01:31:49.009138 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45f29692bb444492a58051889f4f44b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae4e155b9bf04e6ba575e5d36a86fd2b', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 01:31:49.009771 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 01:31:49.010079 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 01:31:49.010414 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 01:31:49.012382 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 01:31:49.023853 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 01:31:49.025439 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa23bf-7ce6-4bfb-902b-c9ce90f20f33 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 01:31:49.035896 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1431/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jul 28 01:31:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 01:31:59.031558 dv-cinder-815582-11-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}